[MediaWiki-commits] [Gerrit] operations/puppet[production]: Fix 'sql' script for multi-instance hosts

2018-01-08 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/397912 )

Change subject: Fix 'sql' script for multi-instance hosts
..


Fix 'sql' script for multi-instance hosts

It needs to extract the port, if one is present, to pass host and port
separately to mysql's `-h` and `--port` options.

Bug: T182713
Change-Id: Icc0fdf43e226f21dbb5e7de7b7cf74f834382cf3
---
M modules/scap/files/sql
1 file changed, 7 insertions(+), 1 deletion(-)

Approvals:
  Tim Starling: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/scap/files/sql b/modules/scap/files/sql
index 05fa44d..f699317 100755
--- a/modules/scap/files/sql
+++ b/modules/scap/files/sql
@@ -68,7 +68,13 @@
 exit 1
 fi
 
+port=3306
+if echo "$host" | grep -qP ':\d+\z'; then
+port=`echo "$host" | sed 's/.*://'`
+host=`echo "$host" | sed 's/:[0-9]\+$//'`
+fi
+
 pass=`echo 'echo \$wgDBadminpassword;' | /usr/local/bin/mwscript eval.php 
--wiki="$lookupdb"`
 
 # Execute mysql.
-exec mysql -u wikiadmin -p$pass -h $host -D $db "$@"
+exec mysql -u wikiadmin -p$pass -h "$host" --port $port -D $db "$@"

-- 
To view, visit https://gerrit.wikimedia.org/r/397912
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Icc0fdf43e226f21dbb5e7de7b7cf74f834382cf3
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Anomie 
Gerrit-Reviewer: Anomie 
Gerrit-Reviewer: Tim Starling 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...luasandbox[master]: Fix README again

2018-01-04 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/402169 )

Change subject: Fix README again
..

Fix README again

Change-Id: Ifcb2445b7fc076ba5f9f20e7c59aba9ca361139f
---
M README
1 file changed, 0 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/php/luasandbox 
refs/changes/69/402169/1

diff --git a/README b/README
index ad31810..144b9ad 100644
--- a/README
+++ b/README
@@ -1,4 +1 @@
-This is a PHP extension. To compile it:
-
 For installation instructions, see 
https://www.mediawiki.org/wiki/Extension:Scribunto#LuaSandbox
-

-- 
To view, visit https://gerrit.wikimedia.org/r/402169
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifcb2445b7fc076ba5f9f20e7c59aba9ca361139f
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/php/luasandbox
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...luasandbox[master]: Update README to remove reference to LuaJIT

2018-01-03 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401894 )

Change subject: Update README to remove reference to LuaJIT
..

Update README to remove reference to LuaJIT

Change-Id: Ic954826ead590583d39870eb0183206db9fcb60b
---
M README
1 file changed, 1 insertion(+), 11 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/php/luasandbox 
refs/changes/94/401894/1

diff --git a/README b/README
index 97e315e..ad31810 100644
--- a/README
+++ b/README
@@ -1,14 +1,4 @@
 This is a PHP extension. To compile it:
 
- $ phpize
- $ ./configure
- $ make
+For installation instructions, see 
https://www.mediawiki.org/wiki/Extension:Scribunto#LuaSandbox
 
-It requires a few dependencies:
-
- luajit - LUA just in time compiler. http://www.luajit.org/
-
-== Mac OS X ==
-
-Using brew:
- brew install luajit

-- 
To view, visit https://gerrit.wikimedia.org/r/401894
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic954826ead590583d39870eb0183206db9fcb60b
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/php/luasandbox
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/vendor[master]: Update wikimedia/remex-html to 1.0.2

2018-01-01 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401416 )

Change subject: Update wikimedia/remex-html to 1.0.2
..

Update wikimedia/remex-html to 1.0.2

Bug: T183379
Change-Id: I9fb9978c69f37bafc0049cd7ed343c033dd50ef8
---
M composer.json
M composer.lock
M composer/installed.json
M wikimedia/remex-html/RemexHtml/Serializer/Formatter.php
M wikimedia/remex-html/RemexHtml/Serializer/Serializer.php
M wikimedia/remex-html/RemexHtml/Tokenizer/PlainAttributes.php
M wikimedia/remex-html/RemexHtml/Tokenizer/TokenHandler.php
M wikimedia/remex-html/RemexHtml/Tokenizer/Tokenizer.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/ActiveFormattingElements.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/CachingStack.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/Dispatcher.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/Element.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/InsertionMode.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/Marker.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/Stack.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/TemplateModeStack.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/TreeBuilder.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/TreeHandler.php
M wikimedia/remex-html/RemexHtml/TreeBuilder/TreeMutationTracer.php
19 files changed, 166 insertions(+), 168 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vendor 
refs/changes/16/401416/1

diff --git a/composer.json b/composer.json
index d50d4f7..4795bf7 100644
--- a/composer.json
+++ b/composer.json
@@ -121,7 +121,7 @@
"wikimedia/php-session-serializer": "1.0.4",
"wikimedia/purtle": "1.0.6",
"wikimedia/relpath": "2.0.0",
-   "wikimedia/remex-html": "1.0.1",
+   "wikimedia/remex-html": "1.0.2",
"wikimedia/running-stat": "1.1.0",
"wikimedia/scoped-callback": "1.0.0",
"wikimedia/testing-access-wrapper": "1.0.0",
diff --git a/composer.lock b/composer.lock
index b52e20e..8e12d0d 100644
--- a/composer.lock
+++ b/composer.lock
@@ -4,7 +4,7 @@
 "Read more about it at 
https://getcomposer.org/doc/01-basic-usage.md#composer-lock-the-lock-file";,
 "This file is @generated automatically"
 ],
-"content-hash": "6f68bf26abea37e7212cc0afe9cb9341",
+"content-hash": "0f3436b07eacbc4f02831120f725d111",
 "packages": [
 {
 "name": "composer/semver",
@@ -3208,16 +3208,16 @@
 },
 {
 "name": "wikimedia/remex-html",
-"version": "1.0.1",
+"version": "1.0.2",
 "source": {
 "type": "git",
 "url": 
"https://github.com/wikimedia/mediawiki-libs-RemexHtml.git";,
-"reference": "3c596e5c66557ed7f37d8a543bc233d7fa80a52f"
+"reference": "c7e8267eabaf1d7e89d65a35571f86ce5bbbc16a"
 },
 "dist": {
 "type": "zip",
-"url": 
"https://api.github.com/repos/wikimedia/mediawiki-libs-RemexHtml/zipball/3c596e5c66557ed7f37d8a543bc233d7fa80a52f";,
-"reference": "3c596e5c66557ed7f37d8a543bc233d7fa80a52f",
+"url": 
"https://api.github.com/repos/wikimedia/mediawiki-libs-RemexHtml/zipball/c7e8267eabaf1d7e89d65a35571f86ce5bbbc16a";,
+"reference": "c7e8267eabaf1d7e89d65a35571f86ce5bbbc16a",
 "shasum": ""
 },
 "require": {
@@ -3225,7 +3225,7 @@
 "wikimedia/utfnormal": "1.1.0"
 },
 "require-dev": {
-"mediawiki/mediawiki-codesniffer": "0.7.2",
+"mediawiki/mediawiki-codesniffer": "0.12.0",
 "phpunit/phpunit": "4.*"
 },
 "type": "library",
@@ -3245,7 +3245,7 @@
 }
 ],
 "description": "Fast HTML 5 parser",
-"time": "2017-03-14T05:28:14+00:00"
+"time": "2018-01-02T00:35:06+00:00"
 },
 {
 "name": "wikimedia/running-stat",
diff --git a/composer/installed.json b/composer/installed.json
index e1d6e49..2c61895 100644
--- a/composer/installed.json
+++ b/composer/installed.json
@@ -1365,49 +1365,6 @@
 "homepage": "https://www.mediawiki.org/wiki/Timestamp";
 },
 {
-"name": "wikimedia/remex-html",
-"version": "1.0.1",
-"version_normalized": "1.0.1.0",
-"source": {
-"type": "git",
-"url": "https://github.com/wikimedia/mediawiki-libs-RemexHtml.git";,
-"reference": "3c596e5c66557ed7f37d8a543bc233d7fa80a52f"
-},
-"dist": {
-"type": "zip",
-"url": 
"https://api.github.com/repos/wikimedia/mediawiki-libs-RemexHtml/zipball/3c596e5c66557ed7f37d8a543bc233d7fa80a52f";,
-"reference": "3c596e5c66557ed7f37d8a543bc233d7fa80a52f

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Split limit report out of Parser::parse()

2017-12-07 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/396303 )

Change subject: Split limit report out of Parser::parse()
..

Split limit report out of Parser::parse()

It was 100 lines. Also update a few nearby comments. The one about just
handling  sections was actually written by Lee, and is
hilariously outdated now.

Change-Id: I12ee2a7e488a3c787b36d3a457c616646aff
---
M includes/parser/Parser.php
1 file changed, 117 insertions(+), 111 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/03/396303/1

diff --git a/includes/parser/Parser.php b/includes/parser/Parser.php
index 3548da9..7b2dc67 100644
--- a/includes/parser/Parser.php
+++ b/includes/parser/Parser.php
@@ -406,13 +406,6 @@
$text, Title $title, ParserOptions $options,
$linestart = true, $clearState = true, $revid = null
) {
-   /**
-* First pass--just handle  sections, pass the rest off
-* to internalParse() which does all the real work.
-*/
-
-   global $wgShowHostnames;
-
if ( $clearState ) {
// We use U+007F DELETE to construct strip markers, so 
we have to make
// sure that this character does not occur in the input 
text.
@@ -474,7 +467,7 @@
}
}
 
-   # Done parsing! Compute runtime adaptive expiry if set
+   # Compute runtime adaptive expiry if set
$this->mOutput->finalizeAdaptiveCacheExpiry();
 
# Warn if too many heavyweight parser functions were used
@@ -485,110 +478,9 @@
);
}
 
-   # Information on include size limits, for the benefit of users 
who try to skirt them
+   # Information on limits, for the benefit of users who try to 
skirt them
if ( $this->mOptions->getEnableLimitReport() ) {
-   $max = $this->mOptions->getMaxIncludeSize();
-
-   $cpuTime = $this->mOutput->getTimeSinceStart( 'cpu' );
-   if ( $cpuTime !== null ) {
-   $this->mOutput->setLimitReportData( 
'limitreport-cputime',
-   sprintf( "%.3f", $cpuTime )
-   );
-   }
-
-   $wallTime = $this->mOutput->getTimeSinceStart( 'wall' );
-   $this->mOutput->setLimitReportData( 
'limitreport-walltime',
-   sprintf( "%.3f", $wallTime )
-   );
-
-   $this->mOutput->setLimitReportData( 
'limitreport-ppvisitednodes',
-   [ $this->mPPNodeCount, 
$this->mOptions->getMaxPPNodeCount() ]
-   );
-   $this->mOutput->setLimitReportData( 
'limitreport-ppgeneratednodes',
-   [ $this->mGeneratedPPNodeCount, 
$this->mOptions->getMaxGeneratedPPNodeCount() ]
-   );
-   $this->mOutput->setLimitReportData( 
'limitreport-postexpandincludesize',
-   [ $this->mIncludeSizes['post-expand'], $max ]
-   );
-   $this->mOutput->setLimitReportData( 
'limitreport-templateargumentsize',
-   [ $this->mIncludeSizes['arg'], $max ]
-   );
-   $this->mOutput->setLimitReportData( 
'limitreport-expansiondepth',
-   [ $this->mHighestExpansionDepth, 
$this->mOptions->getMaxPPExpandDepth() ]
-   );
-   $this->mOutput->setLimitReportData( 
'limitreport-expensivefunctioncount',
-   [ $this->mExpensiveFunctionCount, 
$this->mOptions->getExpensiveParserFunctionLimit() ]
-   );
-   Hooks::run( 'ParserLimitReportPrepare', [ $this, 
$this->mOutput ] );
-
-   $limitReport = "NewPP limit report\n";
-   if ( $wgShowHostnames ) {
-   $limitReport .= 'Parsed by ' . wfHostname() . 
"\n";
-   }
-   $limitReport .= 'Cached time: ' . 
$this->mOutput->getCacheTime() . "\n";
-   $limitReport .= 'Cache expiry: ' . 
$this->mOutput->getCacheExpiry() . "\n";
-   $limitReport .= 'Dynamic content: ' .
-   ( $this->mOutput->hasDynamicContent() ? 'true' 
: 'false' ) .
-   "\n";
-
-   foreach ( $this->mOutput->getLimitReportData() as $key 
=> $value ) {
-   if ( Hooks::run( 'ParserLimitReportFormat',
-   [

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix maintenance script failure when run as a child of a Fast...

2017-12-07 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/396301 )

Change subject: Fix maintenance script failure when run as a child of a FastCGI 
worker
..

Fix maintenance script failure when run as a child of a FastCGI worker

Use PHP_SAPI==cli instead of checking $_SERVER['REQUEST_METHOD'],
since $_SERVER is populated from the environment when running HHVM in
CLI mode. Environment variables set by a FastCGI worker thus leak
through to child processes run via the shell, and cause this check to
fail.

When I wrote this check in March 2004 (r2803), I didn't know about
PHP_SAPI. Checking PHP_SAPI is quite sufficient to prevent web execution,
we use it in other places.

Bug: T111441
Change-Id: Iad8469ee25df4b0e0c2371e7975a300b1695dd8d
---
M maintenance/Maintenance.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/01/396301/1

diff --git a/maintenance/Maintenance.php b/maintenance/Maintenance.php
index 255892b..a3e0ffd 100644
--- a/maintenance/Maintenance.php
+++ b/maintenance/Maintenance.php
@@ -662,7 +662,7 @@
global $IP, $wgCommandLineMode, $wgRequestTime;
 
# Abort if called from a web server
-   if ( isset( $_SERVER ) && isset( $_SERVER['REQUEST_METHOD'] ) ) 
{
+   if ( PHP_SAPI !== 'cli' ) {
$this->fatalError( 'This script must be run from the 
command line' );
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/396301
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iad8469ee25df4b0e0c2371e7975a300b1695dd8d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...namespaceizer[master]: Add a tool to modify and standardize file headers

2017-12-03 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/394905 )

Change subject: Add a tool to modify and standardize file headers
..

Add a tool to modify and standardize file headers

See the core commit Ie0cea0ef5 for the details of the resulting changes.

Change-Id: I4689786bb87ac8fe86564e148d0997cb578dbb47
---
A bin/fixFileHeader.php
M composer.json
A src/FileHeaderFixer.php
3 files changed, 386 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/tools/namespaceizer 
refs/changes/05/394905/1

diff --git a/bin/fixFileHeader.php b/bin/fixFileHeader.php
new file mode 100644
index 000..0334ac9
--- /dev/null
+++ b/bin/fixFileHeader.php
@@ -0,0 +1,29 @@
+ ...\n";
+   exit( 1 );
+   }
+   $files = array_slice( $argv, 1 );
+
+   foreach ( $files as $fileName ) {
+   $input = file_get_contents( $fileName );
+   $result = FileHeaderFixer::fix( $input, function ( $line, $msg 
) use ( $fileName ) {
+   echo "$fileName:$line: $msg\n";
+   } );
+   if ( $result !== false && $input !== $result ) {
+   file_put_contents( $fileName, $result );
+   }
+   }
+}
+
+fixFileHeader( $argv );
diff --git a/composer.json b/composer.json
index 1a2d50d..6fb2e15 100644
--- a/composer.json
+++ b/composer.json
@@ -5,5 +5,10 @@
"psr-4": {
"MediaWiki\\Tool\\Namespaceizer\\": "src/"
}
+   },
+   "minimum-stability": "alpha",
+   "prefer-stable": true,
+   "require": {
+   "nikic/php-parser": "~4.0"
}
 }
diff --git a/src/FileHeaderFixer.php b/src/FileHeaderFixer.php
new file mode 100644
index 000..a7affaa
--- /dev/null
+++ b/src/FileHeaderFixer.php
@@ -0,0 +1,352 @@
+execute();
+   } catch ( Error $error ) {
+   return false;
+   }
+   }
+
+   private function __construct( $text, $errorCallback ) {
+   $this->text = $text;
+   $this->errorCallback = $errorCallback;
+   }
+
+   private function execute() {
+   $lexer = new Lexer\Emulative( [
+   'usedAttributes' => [
+   'comments',
+   'startLine', 'endLine',
+   'startTokenPos', 'endTokenPos',
+   ],
+   ] );
+   $parser = new Parser\Php7( $lexer );
+   $oldAst = $parser->parse( $this->text );
+
+   $traverser = new NodeTraverser;
+   $traverser->addVisitor( new NodeVisitor\CloningVisitor );
+   $newAst = $traverser->traverse( $oldAst );
+
+   if ( count( $newAst ) < 1 ) {
+   $this->fatal( 0, "empty file" );
+   }
+
+   $nonClass = $this->findNonClassCode( $newAst );
+   if ( $nonClass ) {
+   $this->fatal( $nonClass->getLine(), "non-class code 
found" );
+   }
+
+   $firstNode = $newAst[0];
+   $comments = $firstNode->getAttribute( 'comments' );
+   if ( !count( $comments ) ) {
+   $this->error( $firstNode->getLine(), "First node has no 
comments" );
+   return $this->text;
+   }
+
+   $commentText = $comments[0]->getText();
+   $parts = $this->splitComment( $firstNode->getLine(), 
$commentText );
+
+   if ( isset( $parts['start'] ) ) {
+   $parts['start'] = $this->removeCreationDate( 
$parts['start'] );
+   }
+
+   $ingroup = false;
+   if ( isset( $parts['end'] ) ) {
+   $ingroup = $this->getAnnotation( $parts['end'], 
'ingroup' );
+   $parts['end'] = $this->removeAnnotation( $parts['end'], 
'ingroup' );
+   }
+   if ( isset( $parts['license'] ) ) {
+   $newLicenseComment = "/*\n" . $parts['license'] . " 
*/\n";
+   } else {
+   $newLicenseComment = null;
+   }
+
+   if ( isset( $parts['license'] ) && ( isset( $parts['start'] ) 
|| isset( $parts['end'] ) ) ) {
+   $newFileComment = "/**\n";
+   if ( isset( $parts['start'] ) ) {
+   $newFileComment .= $parts['start'];
+   }
+   if ( isset( $parts['end'] ) ) {
+   $newFileComment .= $parts['end'];
+   }
+   $newFileComment .= " */";
+
+   $newComments = [];
+   if ( !$this->isPracticallyEmpty( $newFileComment ) ) {
+   $newComments[] = new Com

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Move license headers out of @file, and other cleanups

2017-12-03 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/394904 )

Change subject: Move license headers out of @file, and other cleanups
..

Move license headers out of @file, and other cleanups

This is the result of running fixFileHeader.php from the
mediawiki/tools/namespaceizer repo on all .php files in the includes
directory. The effect is as follows:

* Don't include the full license header in the @file doc comment, since
  the resulting presentation in Doxygen is unhelpful. Instead, move the
  license header to a non-doc comment. For consistency, also include
  copyright notices which previously appeared above the license header.
* The GPL recommends placing a short description of the file above the
  license header, and many files did do that. These short descriptions
  are also ideal as content for @file doc comments for Doxygen's
  purposes. So, place the pre-existing file descriptions at the top of
  the file, together with @file and other annotations which previously
  below the license.
* Remove @ingroup from @file sections relating to class files. It is
  redundant and unhelpful to list class files alongside the classes they
  contain in Doxygen's module menu. Copied the @ingroup annotation from
  the file comment to any classes in the same file which did not already
  have it, which I think reflects the authors' intents.
* Remove creation dates from file headers. I don't think this
  information is important enough to include in such a prominent place,
  and creation date notices are apparently not protected by the GPL.
* Standardize on having one blank line following the license header.
  It's inconvenient to detect the whitespace environment with
  php-parser, and this was the most common pre-existing style.

Files which had file-level code were skipped, since those are irrelevant
for the PSR-4 migration which motivated this work.

Change-Id: Ie0cea0ef5027c7e583318aed04454bd33e26fcee
---
M includes/AjaxDispatcher.php
M includes/AjaxResponse.php
M includes/AuthPlugin.php
M includes/Autopromote.php
M includes/Block.php
M includes/CategoriesRdf.php
M includes/Category.php
M includes/CategoryFinder.php
M includes/CategoryViewer.php
M includes/CommentStore.php
M includes/CommentStoreComment.php
M includes/DeprecatedGlobal.php
M includes/DerivativeRequest.php
M includes/EditPage.php
M includes/EventRelayerGroup.php
M includes/FauxRequest.php
M includes/Feed.php
M includes/FeedUtils.php
M includes/FileDeleteForm.php
M includes/ForkController.php
M includes/FormOptions.php
M includes/GitInfo.php
M includes/Hooks.php
M includes/Html.php
M includes/HtmlFormatter.php
M includes/LinkFilter.php
M includes/Linker.php
M includes/ListToggle.php
M includes/MWGrants.php
M includes/MWNamespace.php
M includes/MWTimestamp.php
M includes/MagicWord.php
M includes/MagicWordArray.php
M includes/MediaWiki.php
M includes/MergeHistory.php
M includes/Message.php
M includes/MimeMagic.php
M includes/MovePage.php
M includes/OrderedStreamingForkController.php
M includes/OutputPage.php
M includes/PageProps.php
M includes/PathRouter.php
M includes/Pingback.php
M includes/Preferences.php
M includes/PrefixSearch.php
M includes/ProtectionForm.php
M includes/ProxyLookup.php
M includes/RawMessage.php
M includes/Revision.php
M includes/RevisionList.php
M includes/SiteConfiguration.php
M includes/SiteStats.php
M includes/Status.php
M includes/StreamFile.php
M includes/StubObject.php
M includes/TemplatesOnThisPageFormatter.php
M includes/Title.php
M includes/TitleArray.php
M includes/TitleArrayFromResult.php
M includes/TrackingCategories.php
M includes/WebRequest.php
M includes/WebRequestUpload.php
M includes/WebResponse.php
M includes/WikiMap.php
M includes/WikiReference.php
M includes/Xml.php
M includes/XmlJsCode.php
M includes/XmlSelect.php
M includes/actions/Action.php
M includes/actions/CachedAction.php
M includes/actions/CreditsAction.php
M includes/actions/DeleteAction.php
M includes/actions/EditAction.php
M includes/actions/FormAction.php
M includes/actions/FormlessAction.php
M includes/actions/HistoryAction.php
M includes/actions/InfoAction.php
M includes/actions/MarkpatrolledAction.php
M includes/actions/ProtectAction.php
M includes/actions/PurgeAction.php
M includes/actions/RawAction.php
M includes/actions/RenderAction.php
M includes/actions/RevertAction.php
M includes/actions/RollbackAction.php
M includes/actions/SpecialPageAction.php
M includes/actions/SubmitAction.php
M includes/actions/UnprotectAction.php
M includes/actions/UnwatchAction.php
M includes/actions/ViewAction.php
M includes/actions/WatchAction.php
M includes/api/ApiAMCreateAccount.php
M includes/api/ApiAuthManagerHelper.php
M includes/api/ApiBlock.php
M includes/api/ApiCSPReport.php
M includes/api/ApiChangeAuthenticationData.php
M includes/api/ApiCheckToken.php
M includes/api/ApiClearHasMsg.php
M includes/api/ApiClientLogin.php
M includes/api/ApiComparePages.ph

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: A few doc comment fixups

2017-12-03 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/394903 )

Change subject: A few doc comment fixups
..

A few doc comment fixups

* Remove some creation dates, they are not protected by GPL
* Remove duplicate @defgroup API
* Remove @ingroup from some @file doc comments on class files. It is not
  useful to list class files alongside classes in the doxygen module menu.
  Add @ingroup to some more class files that had @ingroup on their file,
  that was probably the author's intent.
* In PackedOverlayImageGallery, use the file comment as a class comment
* Don't put @defgroup and @file in the same comment. @defgroup makes the
  whole doc comment describe the group.
* Instead of putting AnsiTermColorer in two groups, use hierarchical
  groups.

Change-Id: If54f6e0b2bc1ea6de42045885cf836ee67b8e961
---
M includes/api/ApiImport.php
M includes/api/ApiOpenSearch.php
M includes/api/ApiRsd.php
M includes/api/ApiUsageException.php
M includes/deferred/CdnCacheUpdate.php
M includes/gallery/PackedOverlayImageGallery.php
M includes/installer/PhpBugTests.php
M includes/jobqueue/JobSpecification.php
M includes/jobqueue/aggregator/JobQueueAggregator.php
M includes/libs/rdbms/database/IDatabase.php
M includes/libs/rdbms/database/IMaintainableDatabase.php
M includes/libs/rdbms/exception/DBExpectedError.php
M includes/libs/rdbms/loadbalancer/LoadBalancer.php
M includes/specials/SpecialUploadStash.php
M includes/upload/UploadStash.php
M languages/classes/LanguageEn.php
M languages/classes/LanguageGan.php
M maintenance/term/MWTerm.php
18 files changed, 67 insertions(+), 31 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/03/394903/1

diff --git a/includes/api/ApiImport.php b/includes/api/ApiImport.php
index a0f0a8d..822711a 100644
--- a/includes/api/ApiImport.php
+++ b/includes/api/ApiImport.php
@@ -1,9 +1,5 @@
 .@gmail.com"
  *
  * This program is free software; you can redistribute it and/or modify
diff --git a/includes/api/ApiOpenSearch.php b/includes/api/ApiOpenSearch.php
index 419fd14..416fc7f 100644
--- a/includes/api/ApiOpenSearch.php
+++ b/includes/api/ApiOpenSearch.php
@@ -1,7 +1,5 @@
 @gmail.com"
  * Copyright © 2008 Brion Vibber 
  * Copyright © 2014 Wikimedia Foundation and contributors
@@ -382,6 +380,9 @@
}
 }
 
+/**
+ * @ingroup API
+ */
 class ApiOpenSearchFormatJson extends ApiFormatJson {
private $warningsAsError = false;
 
diff --git a/includes/api/ApiRsd.php b/includes/api/ApiRsd.php
index fdc62a8..f20d1c6 100644
--- a/includes/api/ApiRsd.php
+++ b/includes/api/ApiRsd.php
@@ -3,8 +3,6 @@
 /**
  * API for MediaWiki 1.17+
  *
- * Created on October 26, 2010
- *
  * Copyright © 2010 Bryan Tong Minh and Brion Vibber
  *
  * This program is free software; you can redistribute it and/or modify
diff --git a/includes/api/ApiUsageException.php 
b/includes/api/ApiUsageException.php
index 4196add..c200dcb 100644
--- a/includes/api/ApiUsageException.php
+++ b/includes/api/ApiUsageException.php
@@ -16,7 +16,6 @@
  * http://www.gnu.org/copyleft/gpl.html
  *
  * @file
- * @defgroup API API
  */
 
 /**
diff --git a/includes/deferred/CdnCacheUpdate.php 
b/includes/deferred/CdnCacheUpdate.php
index 7fafc0e..301c4f3 100644
--- a/includes/deferred/CdnCacheUpdate.php
+++ b/includes/deferred/CdnCacheUpdate.php
@@ -18,7 +18,6 @@
  * http://www.gnu.org/copyleft/gpl.html
  *
  * @file
- * @ingroup Cache
  */
 
 use Wikimedia\Assert\Assert;
diff --git a/includes/gallery/PackedOverlayImageGallery.php 
b/includes/gallery/PackedOverlayImageGallery.php
index db8ce68..0a5a457 100644
--- a/includes/gallery/PackedOverlayImageGallery.php
+++ b/includes/gallery/PackedOverlayImageGallery.php
@@ -1,8 +1,5 @@
 http://www.gnu.org/copyleft/gpl.html
  *
  * @file
- * @defgroup PHPBugTests PHP known bugs tests
  */
 
 /**
+ * @defgroup PHPBugTests PHP known bugs tests
+ */
+/**
  * Test for PHP+libxml2 bug which breaks XML input subtly with certain 
versions.
  * Known fixed with PHP 5.2.9 + libxml2-2.7.3
  * @see https://bugs.php.net/bug.php?id=45996
diff --git a/includes/jobqueue/JobSpecification.php 
b/includes/jobqueue/JobSpecification.php
index d844795..b62b83c 100644
--- a/includes/jobqueue/JobSpecification.php
+++ b/includes/jobqueue/JobSpecification.php
@@ -18,7 +18,6 @@
  * http://www.gnu.org/copyleft/gpl.html
  *
  * @file
- * @ingroup JobQueue
  */
 
 /**
diff --git a/includes/jobqueue/aggregator/JobQueueAggregator.php 
b/includes/jobqueue/aggregator/JobQueueAggregator.php
index f26beee..433de93 100644
--- a/includes/jobqueue/aggregator/JobQueueAggregator.php
+++ b/includes/jobqueue/aggregator/JobQueueAggregator.php
@@ -158,6 +158,9 @@
}
 }
 
+/**
+ * @ingroup JobQueue
+ */
 class JobQueueAggregatorNull extends JobQueueAggregator {
protected function doNotifyQueueEmpty( $wiki, $type ) {
return true;
diff --git a/includes/libs/rdbms/database/IDatabase.php 

[MediaWiki-commits] [Gerrit] mediawiki...namespaceizer[master]: Added conflict and reserved word detection

2017-11-30 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/392364 )

Change subject: Added conflict and reserved word detection
..


Added conflict and reserved word detection

Detect conflicts due to two classes being renamed to the same thing.
Exempt existing class_alias() calls. Detect class names that use PHP
reserved words.

Change-Id: I91762c902b049fea472a189073acc87ab0ada85f
---
M bin/dumpCoreAliases.php
M src/CoreAliases.php
A src/ReservedWords.php
3 files changed, 333 insertions(+), 117 deletions(-)

Approvals:
  Tim Starling: Verified
  Anomie: Looks good to me, approved



diff --git a/bin/dumpCoreAliases.php b/bin/dumpCoreAliases.php
index e115c2b..f5b88e0 100644
--- a/bin/dumpCoreAliases.php
+++ b/bin/dumpCoreAliases.php
@@ -9,76 +9,94 @@
 
 require __DIR__ . '/../vendor/autoload.php';
 
-$self = array_shift( $argv );
-
-if ( !count( $argv ) ) {
-   echo "Usage: $self \n";
-   exit( 1 );
+function warnCallback( $msg ) {
+   fwrite( STDERR, "FYI: $msg\n" );
 }
 
-if ( $argv[0] === '--counts' ) {
-   $report = 'counts';
-   array_shift( $argv );
-} elseif ( $argv[0] === '--3col' ) {
-   $report = '3col';
-   array_shift( $argv );
-} elseif ( $argv[0] === '--classes' ) {
-   $report = 'classes';
-   array_shift( $argv );
-} else {
-   $report = 'php';
+function errorCallback( $msg ) {
+   fwrite( STDERR, "ERROR: $msg\n" );
 }
 
-if ( !count( $argv ) ) {
-   echo "Usage: $self \n";
-   exit( 1 );
-}
+function dumpCoreAliases() {
+   global $argv;
 
-$file = $argv[0];
+   $self = array_shift( $argv );
 
-$autoload = AutoloadLoader::getAutoloadClasses( $file );
-if ( $autoload === false ) {
-   echo "Unable to open autoload file $file\n";
-   exit( 1 );
-}
+   if ( !count( $argv ) ) {
+   echo "Usage: $self \n";
+   exit( 1 );
+   }
 
-$ca = new CoreAliases( $autoload );
-$aliases = $ca->getAliases();
-
-asort( $aliases );
-
-$counts = [];
-if ( $report === 'php' ) {
-   print " $new ) {
-   if ( $report === 'classes' ) {
-   print "$old\t$new\n";
-   } elseif ( $report === 'php' ) {
-   print "\t'" .
-   strtr( $old, [ "\\" => "", "'" => "\\'" ] ) .
-   "' => '" .
-   strtr( $new, [ "\\" => "", "'" => "\\'" ] ) .
-   "',\n";
+   if ( $argv[0] === '--counts' ) {
+   $report = 'counts';
+   array_shift( $argv );
+   } elseif ( $argv[0] === '--3col' ) {
+   $report = '3col';
+   array_shift( $argv );
+   } elseif ( $argv[0] === '--classes' ) {
+   $report = 'classes';
+   array_shift( $argv );
} else {
-   if ( preg_match( '/(.*)([^]*)$/', $new, $m ) ) {
-   if ( $report === 'counts' ) {
-   if ( !isset( $counts[$m[1]] ) ) {
-   $counts[$m[1]] = 0;
+   $report = 'php';
+   }
+
+   if ( !count( $argv ) ) {
+   echo "Usage: $self \n";
+   exit( 1 );
+   }
+
+   $file = $argv[0];
+
+   $autoload = AutoloadLoader::getAutoloadClasses( $file );
+   if ( $autoload === false ) {
+   echo "Unable to open autoload file $file\n";
+   exit( 1 );
+   }
+
+   $ca = new CoreAliases( $autoload, 'errorCallback', 'warnCallback' );
+   $aliases = $ca->getAliases();
+   
+   if ( $aliases === false ) {
+   exit( 1 );
+   }
+
+   asort( $aliases );
+
+   $counts = [];
+   if ( $report === 'php' ) {
+   print " $new ) {
+   if ( $report === 'classes' ) {
+   print "$old\t$new\n";
+   } elseif ( $report === 'php' ) {
+   print "\t'" .
+   strtr( $old, [ "\\" => "", "'" => "\\'" ] ) 
.
+   "' => '" .
+   strtr( $new, [ "\\" => "", "'" => "\\'" ] ) 
.
+   "',\n";
+   } else {
+   if ( preg_match( '/(.*)([^]*)$/', $new, $m ) ) {
+   if ( $report === 'counts' ) {
+   if ( !isset( $counts[$m[1]] ) ) {
+   $counts[$m[1]] = 0;
+   }
+   $counts[$m[1]]++;
+   } elseif ( $report === '3col' ) {
+   print "$old\t{$m[1]}\t{$m[2]}\n";
}
-   $counts[$m[1]]++;
-   } elseif ( $report === '3col' ) {
-   print "$old\t{$m[1]}

[MediaWiki-commits] [Gerrit] mediawiki...namespaceizer[master]: Added conflict and reserved word detection

2017-11-19 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/392364 )

Change subject: Added conflict and reserved word detection
..

Added conflict and reserved word detection

Detect conflicts due to two classes being renamed to the same thing.
Exempt existing class_alias() calls. Detect class names that use PHP
reserved words.

Change-Id: I91762c902b049fea472a189073acc87ab0ada85f
---
M bin/dumpCoreAliases.php
M src/CoreAliases.php
2 files changed, 241 insertions(+), 117 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/tools/namespaceizer 
refs/changes/64/392364/1

diff --git a/bin/dumpCoreAliases.php b/bin/dumpCoreAliases.php
index e115c2b..b270f53 100644
--- a/bin/dumpCoreAliases.php
+++ b/bin/dumpCoreAliases.php
@@ -9,76 +9,94 @@
 
 require __DIR__ . '/../vendor/autoload.php';
 
-$self = array_shift( $argv );
-
-if ( !count( $argv ) ) {
-   echo "Usage: $self \n";
-   exit( 1 );
+function warnCallback( $msg ) {
+   fwrite( STDERR, "WARNING: $msg\n" );
 }
 
-if ( $argv[0] === '--counts' ) {
-   $report = 'counts';
-   array_shift( $argv );
-} elseif ( $argv[0] === '--3col' ) {
-   $report = '3col';
-   array_shift( $argv );
-} elseif ( $argv[0] === '--classes' ) {
-   $report = 'classes';
-   array_shift( $argv );
-} else {
-   $report = 'php';
+function errorCallback( $msg ) {
+   fwrite( STDERR, "ERROR: $msg\n" );
 }
 
-if ( !count( $argv ) ) {
-   echo "Usage: $self \n";
-   exit( 1 );
-}
+function dumpCoreAliases() {
+   global $argv;
 
-$file = $argv[0];
+   $self = array_shift( $argv );
 
-$autoload = AutoloadLoader::getAutoloadClasses( $file );
-if ( $autoload === false ) {
-   echo "Unable to open autoload file $file\n";
-   exit( 1 );
-}
+   if ( !count( $argv ) ) {
+   echo "Usage: $self \n";
+   exit( 1 );
+   }
 
-$ca = new CoreAliases( $autoload );
-$aliases = $ca->getAliases();
-
-asort( $aliases );
-
-$counts = [];
-if ( $report === 'php' ) {
-   print " $new ) {
-   if ( $report === 'classes' ) {
-   print "$old\t$new\n";
-   } elseif ( $report === 'php' ) {
-   print "\t'" .
-   strtr( $old, [ "\\" => "", "'" => "\\'" ] ) .
-   "' => '" .
-   strtr( $new, [ "\\" => "", "'" => "\\'" ] ) .
-   "',\n";
+   if ( $argv[0] === '--counts' ) {
+   $report = 'counts';
+   array_shift( $argv );
+   } elseif ( $argv[0] === '--3col' ) {
+   $report = '3col';
+   array_shift( $argv );
+   } elseif ( $argv[0] === '--classes' ) {
+   $report = 'classes';
+   array_shift( $argv );
} else {
-   if ( preg_match( '/(.*)([^]*)$/', $new, $m ) ) {
-   if ( $report === 'counts' ) {
-   if ( !isset( $counts[$m[1]] ) ) {
-   $counts[$m[1]] = 0;
+   $report = 'php';
+   }
+
+   if ( !count( $argv ) ) {
+   echo "Usage: $self \n";
+   exit( 1 );
+   }
+
+   $file = $argv[0];
+
+   $autoload = AutoloadLoader::getAutoloadClasses( $file );
+   if ( $autoload === false ) {
+   echo "Unable to open autoload file $file\n";
+   exit( 1 );
+   }
+
+   $ca = new CoreAliases( $autoload, 'errorCallback', 'warnCallback' );
+   $aliases = $ca->getAliases();
+   
+   if ( $aliases === false ) {
+   exit( 1 );
+   }
+
+   asort( $aliases );
+
+   $counts = [];
+   if ( $report === 'php' ) {
+   print " $new ) {
+   if ( $report === 'classes' ) {
+   print "$old\t$new\n";
+   } elseif ( $report === 'php' ) {
+   print "\t'" .
+   strtr( $old, [ "\\" => "", "'" => "\\'" ] ) 
.
+   "' => '" .
+   strtr( $new, [ "\\" => "", "'" => "\\'" ] ) 
.
+   "',\n";
+   } else {
+   if ( preg_match( '/(.*)([^]*)$/', $new, $m ) ) {
+   if ( $report === 'counts' ) {
+   if ( !isset( $counts[$m[1]] ) ) {
+   $counts[$m[1]] = 0;
+   }
+   $counts[$m[1]]++;
+   } elseif ( $report === '3col' ) {
+   print "$old\t{$m[1]}\t{$m[2]}\n";
}
-   $counts[$m[1]]++;
-   } elseif ( $report === '3col' ) {
-   print "$old\t{$m[1]}\t

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix RemexCompatMunger infinite recursion

2017-11-17 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/392028 )

Change subject: Fix RemexCompatMunger infinite recursion
..

Fix RemexCompatMunger infinite recursion

When TreeBuilder requests reparenting of all child nodes of a given
element, we do this by removing the existing child nodes, and then
inserting the proposed new parent under the old parent. However, when a
p-wrap diversion is in place, the insertion of the new parent is
diverted into the p-wrap, and the p-wrap then becomes a child of the new
parent, causing a reference loop, and ultimately infinite recursion in
Serializer.

Instead, divert the entire reparent request to the p-wrap, so that the
new parent is a child of the p-wrap. This makes sense since the new
parent is always a formatting element. The only caller of
reparentChildren(), apart from proxies, is AAA step 17, which reparents
children under the formatting element cloned from the AFE list.

Left in some debug code for next time.

Bug: T178632
Change-Id: Id77d21d99748e94c064ef24c43ee0033de627b8e
---
M includes/tidy/RemexCompatMunger.php
M includes/tidy/RemexMungerData.php
M tests/phpunit/includes/tidy/RemexDriverTest.php
3 files changed, 70 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/28/392028/1

diff --git a/includes/tidy/RemexCompatMunger.php 
b/includes/tidy/RemexCompatMunger.php
index 73bc5f8..fdb8b1e 100644
--- a/includes/tidy/RemexCompatMunger.php
+++ b/includes/tidy/RemexCompatMunger.php
@@ -174,6 +174,10 @@
$length, $sourceStart, $sourceLength );
}
 
+   private function trace( $msg ) {
+   // echo "[RCM] $msg\n";
+   }
+
/**
 * Insert or reparent an element. Create p-wrappers or split the tag 
stack
 * as necessary.
@@ -242,6 +246,7 @@
if ( $under && $parentData->isPWrapper && !$inline ) {
// [B/b] The element is non-inline and the parent is a 
p-wrapper,
// close the parent and insert into its parent instead
+   $this->trace( 'insert B/b' );
$newParent = $this->serializer->getParentNode( $parent 
);
$parent = $newParent;
$parentData = $parent->snData;
@@ -255,12 +260,14 @@
// [CS/b, DS/i] The parent is splittable and the 
current element is
// inline in block context, or if the current element 
is a block
// under a p-wrapper, split the tag stack.
+   $this->trace( $inline ? 'insert DS/i' : 'insert CS/b' );
$newRef = $this->splitTagStack( $newRef, $inline, 
$sourceStart );
$parent = $newRef;
$parentData = $parent->snData;
} elseif ( $under && $parentData->needsPWrapping && $inline ) {
// [A/i] If the element is inline and we are in 
body/blockquote,
// we need to create a p-wrapper
+   $this->trace( 'insert A/i' );
$newRef = $this->insertPWrapper( $newRef, $sourceStart 
);
$parent = $newRef;
$parentData = $parent->snData;
@@ -268,9 +275,12 @@
// [CU/b] If the element is non-inline and (despite 
attempting to
// split above) there is still an ancestor p-wrap, 
disable that
// p-wrap
+   $this->trace( 'insert CU/b' );
$this->disablePWrapper( $parent, $sourceStart );
+   } else {
+   // [A/b, B/i, C/i, D/b, DU/i] insert as normal
+   $this->trace( 'insert normal' );
}
-   // else [A/b, B/i, C/i, D/b, DU/i] insert as normal
 
// An element with element children is a non-blank element
$parentData->nonblankNodeCount++;
@@ -457,6 +467,20 @@
 
public function reparentChildren( Element $element, Element $newParent, 
$sourceStart ) {
$self = $element->userData;
+   if ( $self->snData->childPElement ) {
+   // Reparent under the p-wrapper instead, so that e.g.
+   //   ...
+   // becomes
+   //   
...
+   //
+   // The formatting element should not be the parent of 
the p-wrap.
+   // Without this special case, the insertElement() of 
the  below
+   // would be diverted into the p-wrapper, causing 
infinite recursion
+   // (T178632)
+   $this->reparentChildren( $self->snData->childPElement, 
$newParent, $sourceStart );
+ 

[MediaWiki-commits] [Gerrit] operations/puppet[production]: In furl use /usr/bin/php instead of php5

2017-11-15 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/391748 )

Change subject: In furl use /usr/bin/php instead of php5
..

In furl use /usr/bin/php instead of php5

This FastCGI client script appears to work in HHVM, and probably also
works in PHP 7.

Change-Id: I310b416d2e1917dd9f93f5b171306a4e670d565f
---
M modules/mediawiki/files/furl
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/48/391748/1

diff --git a/modules/mediawiki/files/furl b/modules/mediawiki/files/furl
index ea79101..4670fd5 100755
--- a/modules/mediawiki/files/furl
+++ b/modules/mediawiki/files/furl
@@ -1,4 +1,4 @@
-#!/usr/bin/php5
+#!/usr/bin/php
 https://gerrit.wikimedia.org/r/391748
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I310b416d2e1917dd9f93f5b171306a4e670d565f
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations...dcat[master]: Work around HHVM bug by using XMLWriter::writeAttribute()

2017-11-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/391489 )

Change subject: Work around HHVM bug by using XMLWriter::writeAttribute()
..

Work around HHVM bug by using XMLWriter::writeAttribute()

HHVM has a bug which causes a null namespace URI passed to
writeAttributeNS() to be converted to an empty string before being
passed to libxml2. This causes libxml2 to output an invalid namespace
declaration. So instead, use writeAttribute() to write the prefixed
attribute name, bypassing libxml2's namespace support.

libxml2 keeps a stack of current xmlns declarations, so you would
think it would be possible to declare a namespace on the root element
and then use it in subsequent elements by specifying the same URI each
time. But libxml does not parse xmlns:prefix="..." attributes to add
them to its declaration list, and has no other way to declare
namespaces, so the relevant attributes would have to actually be present
on the root element.

Bug: T117534
Change-Id: Iae8c95adb7824dd9d44849dd54f1ccc67e70e084
---
M DCAT.php
1 file changed, 42 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/dumps/dcat 
refs/changes/89/391489/1

diff --git a/DCAT.php b/DCAT.php
index ce7e6e2..12811a7 100644
--- a/DCAT.php
+++ b/DCAT.php
@@ -163,22 +163,20 @@
);
 
$xml->startElementNS( 'dcat', 'accessURL', null );
-   $xml->writeAttributeNS( 'rdf', 'resource', null, $url );
+   $xml->writeAttribute( 'rdf:resource', $url );
$xml->endElement();
 
$xml->startElementNS( 'dcat', 'downloadURL', null );
-   $xml->writeAttributeNS( 'rdf', 'resource', null, $url );
+   $xml->writeAttribute( 'rdf:resource', $url );
$xml->endElement();
 
$xml->startElementNS( 'dcterms', 'issued', null );
-   $xml->writeAttributeNS( 'rdf', 'datatype', null,
-   'http://www.w3.org/2001/XMLSchema#date' );
+   $xml->writeAttribute( 'rdf:datatype', 
'http://www.w3.org/2001/XMLSchema#date' );
$xml->text( $dump['timestamp'] );
$xml->endElement();
 
$xml->startElementNS( 'dcat', 'byteSize', null );
-   $xml->writeAttributeNS( 'rdf', 'datatype', null,
-   'http://www.w3.org/2001/XMLSchema#decimal' );
+   $xml->writeAttribute( 'rdf:datatype', 
'http://www.w3.org/2001/XMLSchema#decimal' );
$xml->text( $dump['byteSize'] );
$xml->endElement();
 }
@@ -211,7 +209,7 @@
);
}
$xml->startElementNS( 'dcterms', 'description', null );
-   $xml->writeAttributeNS( 'xml', 'lang', null, $langCode 
);
+   $xml->writeAttribute( 'xml:lang', $langCode );
$xml->text( $formatDescription );
$xml->endElement();
}
@@ -253,21 +251,21 @@
array_push( $ids, $id );
 
$xml->startElementNS( 'rdf', 'Description', null );
-   $xml->writeAttributeNS( 'rdf', 'about', null, $id );
+   $xml->writeAttribute( 'rdf:about', $id );
 
$xml->startElementNS( 'rdf', 'type', null );
-   $xml->writeAttributeNS( 'rdf', 'resource', null,
+   $xml->writeAttribute( 'rdf:resource',
'http://www.w3.org/ns/dcat#Distribution' );
$xml->endElement();
 
$xml->startElementNS( 'dcterms', 'license', null );
-   $xml->writeAttributeNS( 'rdf', 'resource', null,
+   $xml->writeAttribute( 'rdf:resource',
$data['config']["$prefix-info"]['license'] );
$xml->endElement();
 
if ( !$isDump ) {
$xml->startElementNS( 'dcat', 'accessURL', null 
);
-   $xml->writeAttributeNS( 'rdf', 'resource', null,
+   $xml->writeAttribute( 'rdf:resource',

$data['config']["$prefix-info"]['accessURL'] );
$xml->endElement();
} else {
@@ -302,7 +300,7 @@
foreach ( $data['i18n'] as $langCode => $langData ) {
if ( array_key_exists( "dataset-$type-title", $langData ) ) {
$xml->startElementNS( 'dcterms', 'title', null );
-   $xml->writeAttributeNS( 'xml', 'lang', null, $langCode 
);
+   $xml->writeAttribute( 'xml:lang', $langCode );
if ( $type === 'live' ) {
$xml->text( $langData['dataset-live-title'] );
} else {
@@ -314,7 +312,7 @@
}
if ( array_key_exists( "dataset-$type-description"

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Do not limit filesize when running a maintenance script

2017-11-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/391170 )

Change subject: Do not limit filesize when running a maintenance script
..

Do not limit filesize when running a maintenance script

Starting HHVM may require writing very large files, so it can't have the
same file size limit as image scaling etc. The memory limit was already
disabled for much the same reason.

This is the only caller of wfShellWikiCmd() in core which proceeds to
call wfShellExec().

Bug: T145819
Change-Id: I1ab35edbbdb63c2d6f5f578cba2547be79a965ef
---
M includes/SiteConfiguration.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/70/391170/1

diff --git a/includes/SiteConfiguration.php b/includes/SiteConfiguration.php
index 7a01a65..2d1d961 100644
--- a/includes/SiteConfiguration.php
+++ b/includes/SiteConfiguration.php
@@ -556,7 +556,7 @@
]
);
// ulimit5.sh breaks this call
-   $data = trim( wfShellExec( $cmd, $retVal, [], [ 
'memory' => 0 ] ) );
+   $data = trim( wfShellExec( $cmd, $retVal, [], [ 
'memory' => 0, 'filesize' => 0 ] ) );
if ( $retVal != 0 || !strlen( $data ) ) {
throw new MWException( "Failed to run 
getConfiguration.php." );
}

-- 
To view, visit https://gerrit.wikimedia.org/r/391170
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1ab35edbbdb63c2d6f5f578cba2547be79a965ef
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...Wikibase[master]: To identify superseded requests, use the requested search te...

2017-11-09 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/390365 )

Change subject: To identify superseded requests, use the requested search term 
instead of the returned search term
..

To identify superseded requests, use the requested search term instead of the 
returned search term

Previously, the search suggestions dropdown would be updated only if the
the search term in the response matched the most recently dispatched
search term. This was supposed to suppress out-of-order responses. However,
it had the undesired effect of suppressing all responses in which the
search term is changed by MediaWiki's Unicode normalisation.

It's not sufficient to run NFC on the client-side, since MediaWiki's
Unicode cleanup routine is not precisely the same as NFC. Also, NFC
changes from one Unicode version to the next. And also, it is slow.

So, use the term precisely as requested to identify responses. It is
already available in the relevant closure.

Bug: T170779
Change-Id: Id0d1d081e921a0f24a380260dcb5186d98e443a9
---
M view/resources/jquery/wikibase/jquery.wikibase.entityselector.js
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Wikibase 
refs/changes/65/390365/1

diff --git a/view/resources/jquery/wikibase/jquery.wikibase.entityselector.js 
b/view/resources/jquery/wikibase/jquery.wikibase.entityselector.js
index c2126b2..55ffada 100644
--- a/view/resources/jquery/wikibase/jquery.wikibase.entityselector.js
+++ b/view/resources/jquery/wikibase/jquery.wikibase.entityselector.js
@@ -293,7 +293,7 @@
 
deferred.resolve(
response.search,
-   response.searchinfo.search,
+   term,
response[ 'search-continue' ]
);
} )

-- 
To view, visit https://gerrit.wikimedia.org/r/390365
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id0d1d081e921a0f24a380260dcb5186d98e443a9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Wikibase
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/puppet[production]: dumps: Fix typo in archive_index.html

2017-11-01 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/387977 )

Change subject: dumps: Fix typo in archive_index.html
..


dumps: Fix typo in archive_index.html

Change-Id: Ia8e3f7ff10b1c01649fee2fcd263b738b8555acf
---
M modules/dumps/files/web/html/archive_index.html
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Tim Starling: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/dumps/files/web/html/archive_index.html 
b/modules/dumps/files/web/html/archive_index.html
index 16469b1..1cb9e9f 100644
--- a/modules/dumps/files/web/html/archive_index.html
+++ b/modules/dumps/files/web/html/archive_index.html
@@ -18,7 +18,7 @@
 

 These dumps, one or two for the various projects for the years listed 
below, are
-provided as historical material only.  For the ealier files, '''do not 
try to import these into a
+provided as historical material only.  For the earlier files, '''do 
not try to import these into a
 current installation of MediaWiki.'''  The dumps are in varying 
formats depending on
 when they were created.  Happy perusing!


-- 
To view, visit https://gerrit.wikimedia.org/r/387977
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia8e3f7ff10b1c01649fee2fcd263b738b8555acf
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Legoktm 
Gerrit-Reviewer: ArielGlenn 
Gerrit-Reviewer: Tim Starling 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/puppet[production]: Update dumps archive_index.html for the files I just uploaded

2017-11-01 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/383958 )

Change subject: Update dumps archive_index.html for the files I just uploaded
..


Update dumps archive_index.html for the files I just uploaded

Change-Id: I6e87761939d8769046ceee062b1628ab28cfe091
---
M modules/dumps/files/web/html/archive_index.html
1 file changed, 1 insertion(+), 0 deletions(-)

Approvals:
  Tim Starling: Looks good to me, approved
  Krinkle: Looks good to me, but someone else must approve
  jenkins-bot: Verified



diff --git a/modules/dumps/files/web/html/archive_index.html 
b/modules/dumps/files/web/html/archive_index.html
index 71ae382..16469b1 100644
--- a/modules/dumps/files/web/html/archive_index.html
+++ b/modules/dumps/files/web/html/archive_index.html
@@ -23,6 +23,7 @@
 when they were created.  Happy perusing!

2001 (UseModWiki) tarballs of the directory
+   2001 (UseModWiki) dump converted to MediaWiki 
XML
2002 (UseModWiki) tarballs of the directory
2003 (phpwiki?) dumps of sql tables
2005 (MediaWiki 1.5?) XML files, warning: old 
schema!

-- 
To view, visit https://gerrit.wikimedia.org/r/383958
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I6e87761939d8769046ceee062b1628ab28cfe091
Gerrit-PatchSet: 3
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Krinkle 
Gerrit-Reviewer: Tim Starling 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/puppet[production]: Update dumps archive_index.html for the files I just uploaded

2017-10-12 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/383958 )

Change subject: Update dumps archive_index.html for the files I just uploaded
..

Update dumps archive_index.html for the files I just uploaded

Change-Id: I6e87761939d8769046ceee062b1628ab28cfe091
---
M modules/dumps/files/web/html/archive_index.html
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/58/383958/1

diff --git a/modules/dumps/files/web/html/archive_index.html 
b/modules/dumps/files/web/html/archive_index.html
index 71ae382..4b4bd83 100644
--- a/modules/dumps/files/web/html/archive_index.html
+++ b/modules/dumps/files/web/html/archive_index.html
@@ -23,6 +23,7 @@
 when they were created.  Happy perusing!

2001 (UseModWiki) tarballs of the directory
+   2001 (UseModWiki) dump converted to MediaWiki 
XML
2002 (UseModWiki) tarballs of the directory
2003 (phpwiki?) dumps of sql tables
2005 (MediaWiki 1.5?) XML files, warning: old 
schema!

-- 
To view, visit https://gerrit.wikimedia.org/r/383958
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6e87761939d8769046ceee062b1628ab28cfe091
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Deprecate $wgEnableAPI and $wgEnableWriteAPI

2017-10-11 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/383726 )

Change subject: Deprecate $wgEnableAPI and $wgEnableWriteAPI
..

Deprecate $wgEnableAPI and $wgEnableWriteAPI

$wgEnableAPI and $wgEnableWriteAPI are now deprecated and will be removed in
a future version. The API is now considered to be stable, secure and
essential.

Bug: T115414
Change-Id: I8c98ce63afd82677ecf662f8159e58be1759bde0
---
M RELEASE-NOTES-1.31
M includes/DefaultSettings.php
2 files changed, 7 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/26/383726/1

diff --git a/RELEASE-NOTES-1.31 b/RELEASE-NOTES-1.31
index 3fd1fc8..57cbec4 100644
--- a/RELEASE-NOTES-1.31
+++ b/RELEASE-NOTES-1.31
@@ -6,7 +6,9 @@
 production.
 
 === Configuration changes in 1.31 ===
-* …
+* $wgEnableAPI and $wgEnableWriteAPI are now deprecated and will be removed in
+  a future version. The API is now considered to be stable, secure and
+  essential.
 
 === New features in 1.31 ===
 * …
diff --git a/includes/DefaultSettings.php b/includes/DefaultSettings.php
index 780976a..bd944d2 100644
--- a/includes/DefaultSettings.php
+++ b/includes/DefaultSettings.php
@@ -7953,6 +7953,8 @@
  * machine-readable data via api.php
  *
  * See https://www.mediawiki.org/wiki/API
+ *
+ * @deprecated since 1.31
  */
 $wgEnableAPI = true;
 
@@ -7960,6 +7962,8 @@
  * Allow the API to be used to perform write operations
  * (page edits, rollback, etc.) when an authorised user
  * accesses it
+ *
+ * @deprecated since 1.31
  */
 $wgEnableWriteAPI = true;
 

-- 
To view, visit https://gerrit.wikimedia.org/r/383726
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8c98ce63afd82677ecf662f8159e58be1759bde0
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...Vector[wmf/1.31.0-wmf.1]: [1.31.0-wmf.1] Fix reversed nav elements when viewing cached...

2017-09-27 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/381161 )

Change subject: [1.31.0-wmf.1] Fix reversed nav elements when viewing cached 
HTML
..

[1.31.0-wmf.1] Fix reversed nav elements when viewing cached HTML

Follow-up to 5d0a07bf9dac66a9b705865b5c675f2be960e4e6.
See T48947#3641618 for a detailed explanation of the
problem this is fixing. Essentially, our CSS has to be
compatible with both new and old HTML to avoid problems
with cached pages on Wikimedia wikis. To do this, we
add a CSS class to the body and make the new CSS only
apply on pages that have this class. This patch should
be reverted when the caches expire.

Bug: T48947
Change-Id: Icf26e6690b59f470765b2634534d18d2df25ee2a
---
M SkinVector.php
M components/personalMenu.less
M components/tabs.less
3 files changed, 38 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/skins/Vector 
refs/changes/61/381161/1

diff --git a/SkinVector.php b/SkinVector.php
index f684891..23dd944 100644
--- a/SkinVector.php
+++ b/SkinVector.php
@@ -47,6 +47,7 @@
if ( $this->vectorConfig->get( 'VectorExperimentalPrintStyles' 
) ) {
$className .= ' vector-experimental-print-styles';
}
+   $className .= ' vector-nav-directionality';
return $className;
}
 
diff --git a/components/personalMenu.less b/components/personalMenu.less
index 979831f..a91b7b3 100644
--- a/components/personalMenu.less
+++ b/components/personalMenu.less
@@ -19,11 +19,16 @@
 
li {
line-height: 1.125em;
+   /* @noflip */
float: left;
margin-left: 0.75em;
margin-top: 0.5em;
font-size: @menu-personal-font-size;
white-space: nowrap;
+
+   .vector-nav-directionality & {
+   float: left;
+   }
}
 }
 
diff --git a/components/tabs.less b/components/tabs.less
index 211d1fd..1b78203 100644
--- a/components/tabs.less
+++ b/components/tabs.less
@@ -9,6 +9,7 @@
 
 /* Namespaces and Views */
 div.vectorTabs {
+   /* @noflip */
float: left;
height: 2.5em;
.background-image('images/tab-break.png');
@@ -16,7 +17,12 @@
background-repeat: no-repeat;
padding-left: 1px;
 
+   .vector-nav-directionality & {
+   float: left;
+   }
+
ul {
+   /* @noflip */
float: left;
height: 100%;
list-style-type: none;
@@ -27,7 +33,12 @@
background-position: right bottom;
background-repeat: no-repeat;
 
+   .vector-nav-directionality & {
+   float: left;
+   }
+
li {
+   /* @noflip */
float: left;
line-height: 1.125em;
/* For IE6, overridden later to display:block by modern 
browsers */
@@ -40,6 +51,10 @@
background-position: bottom left;
background-repeat: repeat-x;
white-space: nowrap;
+
+   .vector-nav-directionality & {
+   float: left;
+   }
}
 
/* IGNORED BY IE6 which doesn't support child selector */
@@ -104,19 +119,31 @@
 
/* Ignored by IE6 which doesn't support child selector */
> a {
+   /* @noflip */
float: left;
display: block;
+
+   .vector-nav-directionality & {
+   float: left;
+   }
}
}
 }
 
 /* Variants and Actions */
 div.vectorMenu {
+   /* @noflip */
direction: ltr;
+   /* @noflip */
float: left;
cursor: pointer;
position: relative;
line-height: 1.125em;
+
+   .vector-nav-directionality & {
+   direction: ltr;
+   float: left;
+   }
 }
 
 body.rtl div.vectorMenu {
@@ -125,6 +152,7 @@
 }
 
 div#mw-head div.vectorMenu h3 {
+   /* @noflip */
float: left;
.background-image('images/tab-break.png');
background-repeat: no-repeat;
@@ -135,6 +163,10 @@
// border. It is necessary for the dropdown (div.vectorMenu div.menu) 
to align well.
padding-right: 1px;
margin-right: -1px;
+
+   .vector-nav-directionality & {
+   float: left;
+   }
 }
 
 div.vectorMenu h3 span {

-- 
To view, visit https://gerrit.wikimedia.org/r/381161
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Icf26e6690b59f470765b2634534d18d2df25ee2a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/sk

[MediaWiki-commits] [Gerrit] mediawiki/core[wmf/1.30.0-wmf.18]: [1.30.0-wmf.18] SECURITY: Fix handling of CommentStore and i...

2017-09-18 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378819 )

Change subject: [1.30.0-wmf.18] SECURITY: Fix handling of CommentStore and 
insertSelect
..

[1.30.0-wmf.18] SECURITY: Fix handling of CommentStore and insertSelect

CommentStore->insert() takes the raw comment, not quoted, and returns
fields appropriate for passing as $a to IDatabase->insert() or $values
to ->update(). Such fields need to be passed through
IDatabase->addQuotes() to be appropriate for passing in $varMap to
IDatabase->insertSelect().

Change-Id: Ie64b279ee7cf9c8c396af385e46c826e0597ab1e
---
M includes/filerepo/file/LocalFile.php
1 file changed, 4 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/19/378819/1

diff --git a/includes/filerepo/file/LocalFile.php 
b/includes/filerepo/file/LocalFile.php
index 4c0dea2..fd0f3f3 100644
--- a/includes/filerepo/file/LocalFile.php
+++ b/includes/filerepo/file/LocalFile.php
@@ -2307,7 +2307,6 @@
 
$encTimestamp = $dbw->addQuotes( $dbw->timestamp( $now ) );
$encUserId = $dbw->addQuotes( $this->user->getId() );
-   $encReason = $dbw->addQuotes( $this->reason );
$encGroup = $dbw->addQuotes( 'deleted' );
$ext = $this->file->getExtension();
$dotExt = $ext === '' ? '' : ".$ext";
@@ -2350,7 +2349,10 @@
];
$joins = [];
 
-   $fields += $commentStoreFaReason->insert( $dbw, 
$encReason );
+   $fields += array_map(
+   [ $dbw, 'addQuotes' ],
+   $commentStoreFaReason->insert( $dbw, 
$this->reason )
+   );
 
if ( $wgCommentTableSchemaMigrationStage <= 
MIGRATION_WRITE_BOTH ) {
$fields['fa_description'] = 'img_description';

-- 
To view, visit https://gerrit.wikimedia.org/r/378819
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie64b279ee7cf9c8c396af385e46c826e0597ab1e
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: wmf/1.30.0-wmf.18
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Anomie 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: SECURITY: Fix handling of CommentStore and insertSelect

2017-09-18 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/378818 )

Change subject: SECURITY: Fix handling of CommentStore and insertSelect
..

SECURITY: Fix handling of CommentStore and insertSelect

CommentStore->insert() takes the raw comment, not quoted, and returns
fields appropriate for passing as $a to IDatabase->insert() or $values
to ->update(). Such fields need to be passed through
IDatabase->addQuotes() to be appropriate for passing in $varMap to
IDatabase->insertSelect().

Change-Id: Ie64b279ee7cf9c8c396af385e46c826e0597ab1e
---
M includes/filerepo/file/LocalFile.php
1 file changed, 4 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/18/378818/1

diff --git a/includes/filerepo/file/LocalFile.php 
b/includes/filerepo/file/LocalFile.php
index 4c0dea2..fd0f3f3 100644
--- a/includes/filerepo/file/LocalFile.php
+++ b/includes/filerepo/file/LocalFile.php
@@ -2307,7 +2307,6 @@
 
$encTimestamp = $dbw->addQuotes( $dbw->timestamp( $now ) );
$encUserId = $dbw->addQuotes( $this->user->getId() );
-   $encReason = $dbw->addQuotes( $this->reason );
$encGroup = $dbw->addQuotes( 'deleted' );
$ext = $this->file->getExtension();
$dotExt = $ext === '' ? '' : ".$ext";
@@ -2350,7 +2349,10 @@
];
$joins = [];
 
-   $fields += $commentStoreFaReason->insert( $dbw, 
$encReason );
+   $fields += array_map(
+   [ $dbw, 'addQuotes' ],
+   $commentStoreFaReason->insert( $dbw, 
$this->reason )
+   );
 
if ( $wgCommentTableSchemaMigrationStage <= 
MIGRATION_WRITE_BOTH ) {
$fields['fa_description'] = 'img_description';

-- 
To view, visit https://gerrit.wikimedia.org/r/378818
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie64b279ee7cf9c8c396af385e46c826e0597ab1e
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Anomie 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Use RemexHtml instead of Tidy on mediawikiwiki, testwiki

2017-09-13 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/377943 )

Change subject: Use RemexHtml instead of Tidy on mediawikiwiki, testwiki
..

Use RemexHtml instead of Tidy on mediawikiwiki, testwiki

Change-Id: I97c6eab07c827fb558b8467f605c3ae3cf4f23dd
---
M wmf-config/InitialiseSettings.php
1 file changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/43/377943/1

diff --git a/wmf-config/InitialiseSettings.php 
b/wmf-config/InitialiseSettings.php
index 0fe9069..b8c17a6 100644
--- a/wmf-config/InitialiseSettings.php
+++ b/wmf-config/InitialiseSettings.php
@@ -19025,6 +19025,12 @@
'donatewiki' => false,
 ],
 
+'wgTidyConfig' => [
+   'default' => null,
+   'mediawikiwiki' => [ 'driver' => 'RemexHtml' ],
+   'testwiki' => [ 'driver' => 'RemexHtml' ],
+],
+
 // Cache ResourceLoader modules in localStorage
 'wgResourceLoaderStorageEnabled' => [
'default' => true,

-- 
To view, visit https://gerrit.wikimedia.org/r/377943
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I97c6eab07c827fb558b8467f605c3ae3cf4f23dd
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Re-enable EtcdConfig in beta cluster

2017-08-31 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/375108 )

Change subject: Re-enable EtcdConfig in beta cluster
..

Re-enable EtcdConfig in beta cluster

Use a single EtcdConfig object, with slashes in key names

This reverts commit b729a21a710dfb82befc364cfaa2ee3b79a398af.

Depends-On: Ica0914e61baba9c0462481925be15d79b66dc342
Change-Id: I36d43b534c6c872566fa2a6aa06508b7d6a0b077
---
M wmf-config/CommonSettings.php
A wmf-config/etcd.php
2 files changed, 43 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/08/375108/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 764ae5b..aa9a196 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -107,9 +107,13 @@
 # Shorthand when we have no master-slave situation to keep into account
 $wmfLocalServices = $wmfAllServices[$wmfDatacenter];
 
-# Master datacenter
-# The datacenter from which we serve traffic.
-$wmfMasterDatacenter = 'eqiad';
+# Labs-only for testing, eventually etcd.php will be used in production as well
+if ( $wmfRealm === 'labs' ) {
+   # Get configuration from etcd. This gives us the correct 
$wmfMasterDatacenter
+   require "$wmfConfigDir/etcd.php";
+} else {
+   $wmfMasterDatacenter = 'eqiad';
+}
 
 $wmfMasterServices = $wmfAllServices[$wmfMasterDatacenter];
 
diff --git a/wmf-config/etcd.php b/wmf-config/etcd.php
new file mode 100644
index 000..9d7b3e0
--- /dev/null
+++ b/wmf-config/etcd.php
@@ -0,0 +1,36 @@
+ $wmfLocalServices['etcd'],
+   'protocol' => 'https',
+   'directory' => "conftool/v1/mediawiki-config",
+   'cache' => $localCache,
+   ] );
+
+   # Read only mode
+   $wgReadOnly = $etcdConfig->get( "$wmfDatacenter/ReadOnly" );
+
+   # Master datacenter
+   # The datacenter from which we serve traffic.
+   $wmfMasterDatacenter = $etcdConfig->get( 'common/WMFMasterDatacenter' );
+}
+
+wmfSetupEtcd();

-- 
To view, visit https://gerrit.wikimedia.org/r/375108
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I36d43b534c6c872566fa2a6aa06508b7d6a0b077
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: EtcdConfig: allow slashes in config key names

2017-08-31 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/375104 )

Change subject: EtcdConfig: allow slashes in config key names
..

EtcdConfig: allow slashes in config key names

Allowing slashes in config key names allows us to trivially support the
proposed hierarchical structure with a single EtcdConfig object, by
fetching values with the relevant prefixes on startup.

Bug: T156924
Change-Id: Ica0914e61baba9c0462481925be15d79b66dc342
---
M autoload.php
M includes/config/EtcdConfig.php
M tests/phpunit/includes/config/EtcdConfigTest.php
3 files changed, 119 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/04/375104/1

diff --git a/autoload.php b/autoload.php
index 3a2ae10..9d205cb 100644
--- a/autoload.php
+++ b/autoload.php
@@ -437,6 +437,7 @@
'EraseArchivedFile' => __DIR__ . '/maintenance/eraseArchivedFile.php',
'ErrorPageError' => __DIR__ . '/includes/exception/ErrorPageError.php',
'EtcdConfig' => __DIR__ . '/includes/config/EtcdConfig.php',
+   'EtcdConfigParseError' => __DIR__ . 
'/includes/config/EtcdConfigParseError.php',
'EventRelayer' => __DIR__ . 
'/includes/libs/eventrelayer/EventRelayer.php',
'EventRelayerGroup' => __DIR__ . '/includes/EventRelayerGroup.php',
'EventRelayerKafka' => __DIR__ . 
'/includes/libs/eventrelayer/EventRelayerKafka.php',
diff --git a/includes/config/EtcdConfig.php b/includes/config/EtcdConfig.php
index d7dc45a..0a6054e 100644
--- a/includes/config/EtcdConfig.php
+++ b/includes/config/EtcdConfig.php
@@ -228,7 +228,7 @@
// Retrieve all the values under the MediaWiki config directory
list( $rcode, $rdesc, /* $rhdrs */, $rbody, $rerr ) = 
$this->http->run( [
'method' => 'GET',
-   'url' => 
"{$this->protocol}://{$address}/v2/keys/{$this->directory}/",
+   'url' => 
"{$this->protocol}://{$address}/v2/keys/{$this->directory}/?recursive=true",
'headers' => [ 'content-type' => 'application/json' ]
] );
 
@@ -240,28 +240,65 @@
empty( $terminalCodes[$rcode] )
];
}
+   try {
+   return [ $this->parseResponse( $rbody ), null, false ];
+   } catch ( EtcdConfigParseError $e ) {
+   return [ null, $e->getMessage(), false ];
+   }
+   }
 
+   /**
+* Parse a response body, throwing EtcdConfigParseError if there is a 
validation error
+*
+* @param string $rbody
+* @return array
+*/
+   protected function parseResponse( $rbody ) {
$info = json_decode( $rbody, true );
-   if ( $info === null || !isset( $info['node']['nodes'] ) ) {
-   return [ null, "Unexpected JSON response; missing 
'nodes' list.", false ];
+   if ( $info === null ) {
+   throw new EtcdConfigParseError( "Error unserializing 
JSON response." );
}
-
+   if ( !isset( $info['node'] ) || !is_array( $info['node'] ) ) {
+   throw new EtcdConfigParseError( "Unexpected JSON 
response: Missing or invalid node at top level." );
+   }
$config = [];
-   foreach ( $info['node']['nodes'] as $node ) {
-   if ( !empty( $node['dir'] ) ) {
-   continue; // skip directories
-   }
+   $this->parseDirectory( '', $info['node'], $config );
+   return $config;
 
-   $name = basename( $node['key'] );
-   $value = $this->unserialize( $node['value'] );
-   if ( !is_array( $value ) || !array_key_exists( 'val', 
$value ) ) {
-   return [ null, "Failed to parse value for 
'$name'.", false ];
-   }
+   }
 
-   $config[$name] = $value['val'];
+   /**
+* Recursively parse a directory node and populate the array passed by
+* reference, throwing EtcdConfigParseError if there is a validation 
error
+*
+* @param string $dirName The relative directory name
+* @param array $dirNode The decoded directory node
+* @param array &$config The output array
+*/
+   protected function parseDirectory( $dirName, $dirNode, &$config ) {
+   if ( !isset( $dirNode['nodes'] ) ) {
+   throw new EtcdConfigParseError(
+   "Unexpected JSON response in dir '$dirName'; 
missing 'nodes' list." );
+   }
+   if ( !is_array( $dirNode['nodes'] ) ) {
+   throw new EtcdConfigParseError(
+

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Revert "Fix link prefix/suffixes around Category and Languag...

2017-08-31 Thread Tim Starling (Code Review)
Hello Subramanya Sastry, C. Scott Ananian, jenkins-bot, Arlolra, Jackmcbarn,

I'd like you to do a code review.  Please visit

https://gerrit.wikimedia.org/r/375095

to review the following change.


Change subject: Revert "Fix link prefix/suffixes around Category and Language 
links."
..

Revert "Fix link prefix/suffixes around Category and Language links."

This reverts commit c66c9aa5352743b2f87bc4627a1be529c5d18535.
Bug: T174639

Change-Id: Ibf6d3780f384ba8edc80bf28c893f1aee8ce28a8
---
M includes/parser/Parser.php
M tests/parser/parserTests.txt
2 files changed, 7 insertions(+), 43 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/95/375095/1

diff --git a/includes/parser/Parser.php b/includes/parser/Parser.php
index 5ef0032..3261002 100644
--- a/includes/parser/Parser.php
+++ b/includes/parser/Parser.php
@@ -2323,11 +2323,8 @@

$this->mOutput->addLanguageLink( $nt->getFullText() );
}
 
-   /**
-* Strip the whitespace interwiki links 
produce, see T10897
-*/
$s = rtrim( $s . $prefix );
-   $s .= rtrim( $trail, "\n" );
+   $s .= trim( $trail, "\n" ) == '' ? '' : 
$prefix . $trail;
continue;
}
 
@@ -2352,11 +2349,7 @@
continue;
}
} elseif ( $ns == NS_CATEGORY ) {
-   /**
-* Strip the whitespace Category links 
produce, see T2087
-*/
-   $s = rtrim( $s . $prefix ); # T2087, 
T87753
-   $s .= rtrim( $trail, "\n" );
+   $s = rtrim( $s . "\n" ); # T2087
 
if ( $wasblank ) {
$sortkey = 
$this->getDefaultSort();
@@ -2368,6 +2361,11 @@
$sortkey = 
$this->getConverterLanguage()->convertCategoryKey( $sortkey );
$this->mOutput->addCategory( 
$nt->getDBkey(), $sortkey );
 
+   /**
+* Strip the whitespace Category links 
produce, see T2087
+*/
+   $s .= trim( $prefix . $trail, "\n" ) == 
'' ? '' : $prefix . $trail;
+
continue;
}
}
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 2fa3cb0..ab79b59 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -15808,26 +15808,6 @@
 !! end
 
 !! test
-9. Categories and newlines: should behave properly with linkprefix (T87753)
-!! options
-language=ar
-!! wikitext
-foo bar
-foo bar
-[[تصنيف:Foo]]
-[[تصنيف:Bar]]
-!! html/php
-foo bar
-foo bar
-
-!! html/parsoid
-foo bar
-foo bar
-
-
-!! end
-
-!! test
 Category links with multiple namespaces
 !! wikitext
 [[Category:Project:Foo]]
@@ -15872,20 +15852,6 @@
 
 !! html/parsoid
 xy
-!! end
-
-!! test
-Link prefix/suffixes aren't applied to language links
-!! options
-parsoid=wt2html
-language=is
-!! wikitext
-x[[es:Foo]]y
-!! html/php
-xy
-
-!! html/parsoid
-xhttp://es.wikipedia.org/wiki/Foo"; 
data-parsoid=""/>y
 !! end
 
 !! test

-- 
To view, visit https://gerrit.wikimedia.org/r/375095
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibf6d3780f384ba8edc80bf28c893f1aee8ce28a8
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Arlolra 
Gerrit-Reviewer: C. Scott Ananian 
Gerrit-Reviewer: Jackmcbarn 
Gerrit-Reviewer: Subramanya Sastry 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Split common pre-setup code out of WebStart/doMaintenance

2017-08-22 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/373174 )

Change subject: Split common pre-setup code out of WebStart/doMaintenance
..

Split common pre-setup code out of WebStart/doMaintenance

Introduce PreConfigSetup.php, which is common file-scope code run before
LocalSettings.php.

I'm not maintaining autoload.ide.php since it supports closed source
software which I don't have, and it apparently needs significant work to
make it not be weird and hacky.

Change-Id: I44ac69b6b00a51d015546b9766d89d1c59749334
---
A includes/PreConfigSetup.php
M includes/WebStart.php
M maintenance/doMaintenance.php
3 files changed, 58 insertions(+), 45 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/74/373174/1

diff --git a/includes/PreConfigSetup.php b/includes/PreConfigSetup.php
new file mode 100644
index 000..823c821
--- /dev/null
+++ b/includes/PreConfigSetup.php
@@ -0,0 +1,56 @@
+http://www.gnu.org/copyleft/gpl.html
+ *
+ *
+ * @file
+ */
+
+if ( !defined( 'MEDIAWIKI' ) ) {
+   // Not an entry point
+   exit( 1 );
+}
+
+// Grab profiling functions
+require_once "$IP/includes/profiler/ProfilerFunctions.php";
+
+// Start the autoloader, so that extensions can derive classes from core files
+require_once "$IP/includes/AutoLoader.php";
+
+// Load up some global defines.
+require_once "$IP/includes/Defines.php";
+
+// Start the profiler
+$wgProfiler = [];
+if ( file_exists( "$IP/StartProfiler.php" ) ) {
+   require "$IP/StartProfiler.php";
+}
+
+// Load default settings
+require_once "$IP/includes/DefaultSettings.php";
+
+// Load global functions
+require_once "$IP/includes/GlobalFunctions.php";
+
+// Load composer's autoloader if present
+if ( is_readable( "$IP/vendor/autoload.php" ) ) {
+   require_once "$IP/vendor/autoload.php";
+}
+
+
diff --git a/includes/WebStart.php b/includes/WebStart.php
index e281b6f..8a58e6f 100644
--- a/includes/WebStart.php
+++ b/includes/WebStart.php
@@ -59,31 +59,7 @@
$IP = realpath( '.' ) ?: dirname( __DIR__ );
 }
 
-# Grab profiling functions
-require_once "$IP/includes/profiler/ProfilerFunctions.php";
-
-# Start the autoloader, so that extensions can derive classes from core files
-require_once "$IP/includes/AutoLoader.php";
-
-# Load up some global defines.
-require_once "$IP/includes/Defines.php";
-
-# Start the profiler
-$wgProfiler = [];
-if ( file_exists( "$IP/StartProfiler.php" ) ) {
-   require "$IP/StartProfiler.php";
-}
-
-# Load default settings
-require_once "$IP/includes/DefaultSettings.php";
-
-# Load global functions
-require_once "$IP/includes/GlobalFunctions.php";
-
-# Load composer's autoloader if present
-if ( is_readable( "$IP/vendor/autoload.php" ) ) {
-   require_once "$IP/vendor/autoload.php";
-}
+require_once "$IP/includes/PreConfigSetup.php";
 
 # Assert that composer dependencies were successfully loaded
 # Purposely no leading \ due to it breaking HHVM RepoAuthorative mode
diff --git a/maintenance/doMaintenance.php b/maintenance/doMaintenance.php
index 53a317a..e87e024 100644
--- a/maintenance/doMaintenance.php
+++ b/maintenance/doMaintenance.php
@@ -55,26 +55,7 @@
 // to $maintenance->mSelf. Keep that here for b/c
 $self = $maintenance->getName();
 
-# Start the autoloader, so that extensions can derive classes from core files
-require_once "$IP/includes/AutoLoader.php";
-# Grab profiling functions
-require_once "$IP/includes/profiler/ProfilerFunctions.php";
-
-# Start the profiler
-$wgProfiler = [];
-if ( file_exists( "$IP/StartProfiler.php" ) ) {
-   require "$IP/StartProfiler.php";
-}
-
-// Some other requires
-require_once "$IP/includes/Defines.php";
-require_once "$IP/includes/DefaultSettings.php";
-require_once "$IP/includes/GlobalFunctions.php";
-
-# Load composer's autoloader if present
-if ( is_readable( "$IP/vendor/autoload.php" ) ) {
-   require_once "$IP/vendor/autoload.php";
-}
+require_once "$IP/includes/PreConfigSetup.php";
 
 if ( defined( 'MW_CONFIG_CALLBACK' ) ) {
# Use a callback function to configure MediaWiki

-- 
To view, visit https://gerrit.wikimedia.org/r/373174
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I44ac69b6b00a51d015546b9766d89d1c59749334
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...namespaceizer[master]: [WIP] Core alias list

2017-08-18 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/372521 )

Change subject: [WIP] Core alias list
..

[WIP] Core alias list

Bug: T166010
Change-Id: I05949673837e797093da7b4e4241c00077f912ec
---
A .gitignore
A .gitreview
A bin/dumpCoreAliases.php
A composer.json
A src/AutoloadLoader.php
A src/CoreAliases.php
6 files changed, 513 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/tools/namespaceizer 
refs/changes/21/372521/1

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 000..4719332
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,5 @@
+*~
+.*.swp
+/vendor
+/composer.lock
+/composer.local.json
diff --git a/.gitreview b/.gitreview
new file mode 100644
index 000..dc4e124
--- /dev/null
+++ b/.gitreview
@@ -0,0 +1,6 @@
+[gerrit]
+host=gerrit.wikimedia.org
+port=29418
+project=mediawiki/tools/namespaceizer.git
+defaultbranch=master
+defaultrebase=0
diff --git a/bin/dumpCoreAliases.php b/bin/dumpCoreAliases.php
new file mode 100644
index 000..4c2f982
--- /dev/null
+++ b/bin/dumpCoreAliases.php
@@ -0,0 +1,84 @@
+\n";
+   exit( 1 );
+}
+
+if ( $argv[0] === '--counts' ) {
+   $report = 'counts';
+   array_shift( $argv );
+} elseif ( $argv[0] === '--3col' ) {
+   $report = '3col';
+   array_shift( $argv );
+} elseif ( $argv[0] === '--classes' ) {
+   $report = 'classes';
+   array_shift( $argv );
+} else {
+   $report = 'php';
+}
+
+if ( !count( $argv ) ) {
+   echo "Usage: $self \n";
+   exit( 1 );
+}
+
+$file = $argv[0];
+
+$autoload = AutoloadLoader::getAutoloadClasses( $file );
+if ( $autoload === false ) {
+   echo "Unable to open autoload file $file\n";
+   exit( 1 );
+}
+
+$ca = new CoreAliases( $autoload );
+$aliases = $ca->getAliases();
+
+asort( $aliases );
+
+$counts = [];
+if ( $report === 'php' ) {
+   print " $new ) {
+   if ( $report === 'classes' ) {
+   print "$old\t$new\n";
+   } elseif ( $report === 'php' ) {
+   print "\t'" .
+   strtr( $old, [ "\\" => "", "'" => "\\" ] ) .
+   "' => '" .
+   strtr( $new, [ "\\" => "", "'" => "\\" ] ) .
+   "',\n";
+   } else {
+   if ( preg_match( '/(.*)([^]*)$/', $new, $m ) ) {
+   if ( $report === 'counts' ) {
+   if ( !isset( $counts[$m[1]] ) ) {
+   $counts[$m[1]] = 0;
+   }
+   $counts[$m[1]]++;
+   } elseif ( $report === '3col' ) {
+   print "$old\t{$m[1]}\t{$m[2]}\n";
+   }
+   }
+   }
+}
+if ( $report === 'php' ) {
+   print "];\n";
+}
+
+if ( $report === 'counts' ) {
+   foreach ( $counts as $ns => $count ) {
+   print "$ns\t$count\n";
+   }
+}
diff --git a/composer.json b/composer.json
new file mode 100644
index 000..1a2d50d
--- /dev/null
+++ b/composer.json
@@ -0,0 +1,9 @@
+{
+   "name": "mediawiki/namespaceizer",
+   "description": "Namespace migration script",
+   "autoload": {
+   "psr-4": {
+   "MediaWiki\\Tool\\Namespaceizer\\": "src/"
+   }
+   }
+}
diff --git a/src/AutoloadLoader.php b/src/AutoloadLoader.php
new file mode 100644
index 000..ea04c06
--- /dev/null
+++ b/src/AutoloadLoader.php
@@ -0,0 +1,16 @@
+ 'MediaWiki',
+   'includes/actions' => 'MediaWiki\Action',
+   'includes/api' => null,
+   'includes/auth' => 'MediaWiki\Auth',
+   'includes/cache' => 'MediaWiki', // to parent
+   'includes/cache/localisation' => 
'MediaWiki\Language\LocalisationCache',
+   'includes/changes' => 'MediaWiki\Changes',
+   'includes/changetags' => 'MediaWiki\ChangeTags',
+   'includes/clientpool' => 'MediaWiki', // to parent
+   'includes/collation' => 'MediaWiki\Collation',
+   'includes/compat' => 'MediaWiki', // to parent
+   'includes/compat/normal' => 'MediaWiki', // to parent
+   'includes/composer' => null,
+   'includes/config' => 'MediaWiki\Config',
+   'includes/content' => 'MediaWiki\Content',
+   'includes/context' => 'MediaWiki\Context',
+   'includes/dao' => 'MediaWiki\Dao',
+   'includes/db' => 'MediaWiki\Db',
+   'includes/debug' => 'MediaWiki', // to parent
+   'includes/debug/logger' => 'MediaWiki\Logger', // to parent
+   'includes/debug/logger/monolog' => 'MediaWiki\Logger\Monolog', 
// to parent
+   'includes/deferred' => 'MediaWiki\Deferred',
+   'includes/diff' => 'MediaWiki\Diff',
+   'includes/edit' => 

[MediaWiki-commits] [Gerrit] mediawiki...PoolCounter[master]: Add README.md

2017-07-26 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/368126 )

Change subject: Add README.md
..


Add README.md

Change-Id: Iad03c9ef47cef99056f19bbaf1672952399279d7
---
A README.md
1 file changed, 35 insertions(+), 0 deletions(-)

Approvals:
  Tim Starling: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/README.md b/README.md
new file mode 100644
index 000..5fd0114
--- /dev/null
+++ b/README.md
@@ -0,0 +1,35 @@
+# PoolCounter
+
+## Contributing
+
+### MediaWiki extension
+
+Test the PHP code in PHP via [Composer](https://getcomposer.org/).
+
+$ composer install
+$ composer test
+
+
+Lint interface messages in Node.js via [npm](https://www.npmjs.com/get-npm):
+
+$ npm install
+$ npm test
+
+
+### C Daemon
+
+Build using Make. Requires libevent to be installed.
+(`libevent-dev` package on Debian-based systems.)
+
+$ cd daemon/
+$ make install
+
+
+Test the C code in Ruby with Cucumber. Install using 
[Bundler](https://bundler.io/):
+
+$ gem install bundler
+$ bundle install
+
+$ cd daemon/
+$ make test
+

-- 
To view, visit https://gerrit.wikimedia.org/r/368126
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Iad03c9ef47cef99056f19bbaf1672952399279d7
Gerrit-PatchSet: 4
Gerrit-Project: mediawiki/extensions/PoolCounter
Gerrit-Branch: master
Gerrit-Owner: Krinkle 
Gerrit-Reviewer: Krinkle 
Gerrit-Reviewer: Tim Starling 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...PoolCounter[master]: Show the hostname in error messages

2017-07-21 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/366983 )

Change subject: Show the hostname in error messages
..

Show the hostname in error messages

It's helpful for the system administrator to know which PC server is
giving connection errors and the like.

Error messages from the core will not give the hostname, but they are
less likely to be specific to a PC server.

Bug: T65027
Change-Id: Ic8bfc927b6fa6086b7ae9cc694e848f5808ff58b
---
M PoolCounterClient_body.php
M PoolCounter_ConnectionManager.php
M i18n/en.json
M i18n/qqq.json
4 files changed, 24 insertions(+), 17 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/PoolCounter 
refs/changes/83/366983/1

diff --git a/PoolCounterClient_body.php b/PoolCounterClient_body.php
index 52066e9..07a51c8 100644
--- a/PoolCounterClient_body.php
+++ b/PoolCounterClient_body.php
@@ -8,6 +8,11 @@
private $conn;
 
/**
+* @var string The server host name
+*/
+   private $hostName;
+
+   /**
 * @var PoolCounter_ConnectionManager
 */
static private $manager;
@@ -35,7 +40,8 @@
if ( !$status->isOK() ) {
return $status;
}
-   $this->conn = $status->value;
+   $this->conn = $status->value['conn'];
+   $this->hostName = $status->value['hostName'];
 
// Set the read timeout to be 1.5 times the pool 
timeout.
// This allows the server to time out gracefully before 
we give up on it.
@@ -58,11 +64,11 @@
$conn = $status->value;
wfDebug( "Sending pool counter command: $cmd\n" );
if ( fwrite( $conn, "$cmd\n" ) === false ) {
-   return Status::newFatal( 'poolcounter-write-error' );
+   return Status::newFatal( 'poolcounter-write-error', 
$this->hostName );
}
$response = fgets( $conn );
if ( $response === false ) {
-   return Status::newFatal( 'poolcounter-read-error' );
+   return Status::newFatal( 'poolcounter-read-error', 
$this->hostName );
}
$response = rtrim( $response, "\r\n" );
wfDebug( "Got pool counter response: $response\n" );
@@ -85,7 +91,7 @@
default:
$parts = explode( ' ', $parts[1], 2 );
$errorMsg = isset( $parts[1] ) ? $parts[1] : 
'(no message given)';
-   return Status::newFatal( 
'poolcounter-remote-error', $errorMsg );
+   return Status::newFatal( 
'poolcounter-remote-error', $errorMsg, $this->hostName );
}
return Status::newGood( constant( "PoolCounter::$responseType" 
) );
}
diff --git a/PoolCounter_ConnectionManager.php 
b/PoolCounter_ConnectionManager.php
index a1f5b62..91df439 100644
--- a/PoolCounter_ConnectionManager.php
+++ b/PoolCounter_ConnectionManager.php
@@ -29,12 +29,13 @@
$hashes[$hostName] = md5( $hostName . $key );
}
asort( $hashes );
-   $errno = $errstr = '';
+   $errno = $errstr = $hostName = '';
$conn = null;
foreach ( $hashes as $hostName => $hash ) {
if ( isset( $this->conns[$hostName] ) ) {
$this->refCounts[$hostName]++;
-   return Status::newGood( $this->conns[$hostName] 
);
+   return Status::newGood(
+   [ 'conn' => $this->conns[$hostName], 
'hostName' => $hostName ] );
}
$parts = explode( ':', $hostName, 2 );
if ( count( $parts ) < 2 ) {
@@ -48,12 +49,12 @@
}
}
if ( !$conn ) {
-   return Status::newFatal( 
'poolcounter-connection-error', $errstr );
+   return Status::newFatal( 
'poolcounter-connection-error', $errstr, $hostName );
}
wfDebug( "Connected to pool counter server: $hostName\n" );
$this->conns[$hostName] = $conn;
$this->refCounts[$hostName] = 1;
-   return Status::newGood( $conn );
+   return Status::newGood( [ 'conn' => $conn, 'hostName' => 
$hostName ] );
}
 
/**
diff --git a/i18n/en.json b/i18n/en.json
index 2190e6a..10e180a 100644
--- a/i18n/en.json
+++ b/i18n/en.json
@@ -5,8 +5,8 @@
]
},
"poolcounter-desc": "MediaWiki client for the pool counter daemon",
-   "poolcounter-connection-er

[MediaWiki-commits] [Gerrit] mediawiki...PoolCounter[master]: Fix hour count in stats

2017-07-20 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/366789 )

Change subject: Fix hour count in stats
..

Fix hour count in stats

Change-Id: If886a9ac864ae063c1a121840fbb80f88b6ab1b7
---
M daemon/stats.c
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/PoolCounter 
refs/changes/89/366789/1

diff --git a/daemon/stats.c b/daemon/stats.c
index 279feba..5eea602 100644
--- a/daemon/stats.c
+++ b/daemon/stats.c
@@ -23,6 +23,7 @@
int hours = minutes / 60;
minutes %= 60;
unsigned int days = hours / 24;
+   hours %= 60;
 
int n;
n = sprintf( stats_buffer, "uptime: %u days, %dh %dm %ds\n", days, 
hours, minutes, seconds );

-- 
To view, visit https://gerrit.wikimedia.org/r/366789
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If886a9ac864ae063c1a121840fbb80f88b6ab1b7
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/PoolCounter
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...luasandbox[debian]: Updates for release of 2.0.13

2017-07-20 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/366785 )

Change subject: Updates for release of 2.0.13
..

Updates for release of 2.0.13

* Changelog for branch merge
* Non-alpha version number

Change-Id: Ib09ac1672d95bc6b17521c9e2578a50fce0b9060
---
M debian/changelog
M luasandbox_version.h
2 files changed, 10 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/php/luasandbox 
refs/changes/85/366785/1

diff --git a/debian/changelog b/debian/changelog
index dd918b9..ac5fd70 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,14 +1,21 @@
-php-luasandbox (2.0.13) UNRELEASED; urgency=medium
+php-luasandbox (2.0.13) trusty-wikimedia; urgency=medium
 
   [ Kunal Mehta ]
   * Add COPYING file
+  * Don't try to automatically get the version from the debian/changelog
+  * Remove EXPERIMENTAL
 
   [ Brad Jorsch ]
   * Make tests pass under HHVM
   * Make LuaSandboxFunction_construct.phpt more robust
   * Fix memory leak
+  * Update for PHP 7
+  * Fix HHVM test script
+  * Fix getProfilerFunctionReport 'seconds' and 'percent' output in PHP7
+  * Fix getProfilerFunctionReport sorting in HHVM
+  * Redo PHP->Lua data structure recursion protection
 
- -- Kunal Mehta   Mon, 11 Jul 2016 15:18:54 -0700
+ -- Tim Starling   Fri, 21 Jul 2017 14:08:22 +1000
 
 php-luasandbox (2.0.12) trusty-wikimedia; urgency=medium
 
diff --git a/luasandbox_version.h b/luasandbox_version.h
index 8f1e6c8..7b4589e 100644
--- a/luasandbox_version.h
+++ b/luasandbox_version.h
@@ -1 +1 @@
-#define LUASANDBOX_VERSION "2.0.13-alpha"
+#define LUASANDBOX_VERSION "2.0.13"

-- 
To view, visit https://gerrit.wikimedia.org/r/366785
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib09ac1672d95bc6b17521c9e2578a50fce0b9060
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/php/luasandbox
Gerrit-Branch: debian
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...luasandbox[debian]: Merge branch 'master' into debian

2017-07-20 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/366782 )

Change subject: Merge branch 'master' into debian
..

Merge branch 'master' into debian

Change-Id: Icbacb6818484f81109236ef99bfbb7e713614971
---
M debian/changelog
A debian/compat
A debian/control
A debian/copyright
A debian/php-luasandbox.install
A debian/php-luasandbox.postinst
A debian/php-luasandbox.prerm
A debian/rules
A debian/source/format
9 files changed, 160 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/php/luasandbox 
refs/changes/82/366782/1

diff --git a/debian/changelog b/debian/changelog
index 917e5a9..dd918b9 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,4 +1,3 @@
-<<< HEAD   (b4a80e Add gbp.conf for non-standard branches)
 php-luasandbox (2.0.13) UNRELEASED; urgency=medium
 
   [ Kunal Mehta ]
@@ -163,5 +162,3 @@
   * Initial release
 
  -- Tim Starling   Tue, 14 Aug 2012 11:23:40 +1000
-===
->>> BRANCH (a07565 Redo PHP->Lua data structure recursion protection)
diff --git a/debian/compat b/debian/compat
new file mode 100644
index 000..ec63514
--- /dev/null
+++ b/debian/compat
@@ -0,0 +1 @@
+9
diff --git a/debian/control b/debian/control
new file mode 100644
index 000..dbea8bb
--- /dev/null
+++ b/debian/control
@@ -0,0 +1,32 @@
+Source: php-luasandbox
+Section: web
+Priority: optional
+Maintainer: Tim Starling 
+Build-Depends: debhelper (>= 9),
+   liblua5.1-0-dev,
+   pkg-config,
+   php5-dev (<< 5.5) | php5-dev (>= 5.5),
+   php5-dev (<< 5.5) | hhvm,
+   php5-dev (<< 5.5) | hhvm-dev,
+   php5-dev (<< 5.5) | cmake,
+   php5-dev (<< 5.5) | libboost-dev,
+   php5-dev (<< 5.5) | libgoogle-glog-dev,
+   php5-dev (<< 5.5) | libdouble-conversion-dev,
+   php5-dev (<< 5.5) | libjemalloc-dev,
+   php5-dev (<< 5.5) | libtbb-dev
+Standards-Version: 3.9.5
+Homepage: https://www.mediawiki.org/wiki/Extension:Scribunto
+
+Package: php-luasandbox
+Architecture: any
+Depends: php5 | php5-cli, ${shlibs:Depends}, ${misc:Depends}, ${php:Depends}
+Description: Lua extension for PHP
+ A PHP extension providing a sandboxed Lua environment which can be used to run
+ untrusted code.
+
+Package: hhvm-luasandbox
+Architecture: any
+Depends: ${hhvm:ApiVersion}, ${shlibs:Depends}, ${misc:Depends}
+Description: Lua extension for HHVM
+ A HHVM extension providing a sandboxed Lua environment which can be used to 
run
+ untrusted code.
diff --git a/debian/copyright b/debian/copyright
new file mode 100644
index 000..9b58d8d
--- /dev/null
+++ b/debian/copyright
@@ -0,0 +1,25 @@
+Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: luasandbox
+Source: https://www.mediawiki.org/wiki/Extension:Scribunto
+
+Files: *
+Copyright: 2011-2014 Tim Starling
+   2011-2014 Wikimedia Foundation
+License: Expat
+ Permission is hereby granted, free of charge, to any person obtaining a
+ copy of this software and associated documentation files (the "Software"),
+ to deal in the Software without restriction, including without limitation
+ the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ and/or sell copies of the Software, and to permit persons to whom the
+ Software is furnished to do so, subject to the following conditions:
+ .
+ The above copyright notice and this permission notice shall be included in
+ all copies or substantial portions of the Software.
+ .
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ DEALINGS IN THE SOFTWARE.
diff --git a/debian/php-luasandbox.install b/debian/php-luasandbox.install
new file mode 100644
index 000..e50171f
--- /dev/null
+++ b/debian/php-luasandbox.install
@@ -0,0 +1 @@
+luasandbox.ini /etc/php5/mods-available
diff --git a/debian/php-luasandbox.postinst b/debian/php-luasandbox.postinst
new file mode 100755
index 000..639753a
--- /dev/null
+++ b/debian/php-luasandbox.postinst
@@ -0,0 +1,10 @@
+#!/bin/sh
+
+set -e
+
+#EXTRA#
+[ "$1" = "configure" ] && [ -x /usr/sbin/php5enmod ] && php5enmod -s ALL 
luasandbox
+
+#DEBHELPER#
+
+exit 0
diff --git a/debian/php-luasandbox.prerm b/debian/php-luasandbox.prerm
new file mode 100755
index 000..f9c4bc4
--- /dev/null
+++ b/debian/php-luasandbox.prerm
@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -e
+
+#EXTRA#
+
+if [ "$1" = "remove" ] || [ "$1" = "deconfigure" ]; then
+   [ -x /usr/sbin/php5enmod ] && php5dismod -s ALL luasandbox
+fi;
+
+#DEBHELPER#
+
+exit 0
diff --git a/debian/rules b/debian/rules
new file mode 100755
i

[MediaWiki-commits] [Gerrit] mediawiki...ParserMigration[master]: Add API to ParserMigration

2017-06-29 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/362331 )

Change subject: Add API to ParserMigration
..

Add API to ParserMigration

Bug: T162275
Change-Id: I1f2be6fcacd9ec8be8a3f2cca3c929e555aea852
---
M extension.json
A includes/ApiParserMigration.php
A includes/Mechanism.php
M includes/MigrationEditPage.php
4 files changed, 96 insertions(+), 33 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ParserMigration 
refs/changes/31/362331/1

diff --git a/extension.json b/extension.json
index 0f9c888..94884c3 100644
--- a/extension.json
+++ b/extension.json
@@ -7,13 +7,18 @@
"license-name": "CC0-1.0",
"manifest_version": 1,
"AutoloadClasses": {
+   "MediaWiki\\ParserMigration\\ApiParserMigration": 
"includes/ApiParserMigration.php",
"MediaWiki\\ParserMigration\\EditAction": 
"includes/EditAction.php",
"MediaWiki\\ParserMigration\\MigrationEditPage": 
"includes/MigrationEditPage.php",
-   "MediaWiki\\ParserMigration\\Hooks": "includes/Hooks.php"
+   "MediaWiki\\ParserMigration\\Hooks": "includes/Hooks.php",
+   "MediaWiki\\ParserMigration\\Mechanism": 
"includes/Mechanism.php"
},
"Actions": {
"parsermigration-edit": "MediaWiki\\ParserMigration\\EditAction"
},
+   "APIModules": {
+   "parser-migration": 
"MediaWiki\\ParserMigration\\ApiParserMigration"
+   },
"MessagesDirs": {
"ParserMigration": "i18n"
},
diff --git a/includes/ApiParserMigration.php b/includes/ApiParserMigration.php
new file mode 100644
index 000..5e2e4e5
--- /dev/null
+++ b/includes/ApiParserMigration.php
@@ -0,0 +1,40 @@
+extractRequestParams();
+   $page = $params['page'];
+   $title = \Title::newFromText( $page );
+   if ( !$title ) {
+   $this->dieWithError( 'apierror-missingtitle' );
+   }
+   $revision = \Revision::newFromTitle( $title );
+   $content = $revision->getContent();
+   if ( !$content ) {
+   $this->dieWithError( 'apierror-missingcontent-pageid' );
+   }
+   $mechanism = new Mechanism( $this->getConfig()->get( 
'ParserMigrationTidiers' ) );
+   $user = $this->getUser();
+   $options = \ParserOptions::newCanonical();
+   $outputs = $mechanism->parse( $content, $title, $options, $user 
);
+
+   $result = $this->getResult();
+   $result->addValue( null, 'old', $outputs[0]->getText() );
+   $result->addValue( null, 'new', $outputs[1]->getText() );
+   }
+
+   public function isInternal() {
+   return true;
+   }
+
+   public function getAllowedParams() {
+   return [
+   'page' => [
+   \ApiBase::PARAM_TYPE => 'string',
+   \ApiBase::PARAM_REQUIRED => true,
+   ]
+   ];
+   }
+}
diff --git a/includes/Mechanism.php b/includes/Mechanism.php
new file mode 100644
index 000..41f46c4
--- /dev/null
+++ b/includes/Mechanism.php
@@ -0,0 +1,40 @@
+tidiers = $tidiers;
+   }
+
+   public function parse( \Content $content, \Title $title,
+   \ParserOptions $baseOptions, \User $user
+   ) {
+   $options = clone $baseOptions;
+   $options->setTidy( false );
+   $scopedCallback = $options->setupFakeRevision( $title, 
$content, $user );
+   $parserOutput = $content->getParserOutput( $title, null, 
$options );
+   \ScopedCallback::consume( $scopedCallback );
+
+   $leftOutput = $this->tidyParserOutput( $parserOutput, 
$this->tidiers[0] );
+   $rightOutput = $this->tidyParserOutput( $parserOutput, 
$this->tidiers[1] );
+
+   return [ $leftOutput, $rightOutput ];
+   }
+   /**
+* @param \ParserOutput $parserOutput
+* @param array $config
+* @return \ParserOutput
+*/
+   protected function tidyParserOutput( $parserOutput, $config ) {
+   $tidier = \MWTidy::factory( $config );
+   $newOutput = clone $parserOutput;
+   $newOutput->setText( $tidier->tidy( $newOutput->getRawText() ) 
);
+   return $newOutput;
+   }
+}
diff --git a/includes/MigrationEditPage.php b/includes/MigrationEditPage.php
index a42ad14..93189ae 100644
--- a/includes/MigrationEditPage.php
+++ b/includes/MigrationEditPage.php
@@ -24,53 +24,31 @@
return true;
}
 
-   protected function getPreviewParserOptions() {
-   $parserOptions = parent::getPreviewParserOptions();
-   $parserOptions->setTidy( false );
-   retu

[MediaWiki-commits] [Gerrit] mediawiki...Wigo3[master]: Add .gitreview

2017-06-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/359108 )

Change subject: Add .gitreview
..

Add .gitreview

Change-Id: I1346fdb941be02d37963d7d35f1833a96b9a5db9
---
A .gitreview
1 file changed, 3 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Wigo3 
refs/changes/08/359108/1

diff --git a/.gitreview b/.gitreview
new file mode 100644
index 000..7a3e1de
--- /dev/null
+++ b/.gitreview
@@ -0,0 +1,3 @@
+[gerrit]
+host=gerrit.wikimedia.org
+project=mediawiki/extensions/Wigo3

-- 
To view, visit https://gerrit.wikimedia.org/r/359108
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1346fdb941be02d37963d7d35f1833a96b9a5db9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Wigo3
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...Wigo3[master]: Fix security vulnerabilities

2017-06-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/359107 )

Change subject: Fix security vulnerabilities
..

Fix security vulnerabilities

Removed some commented-out code so that I didn't have to remediate it.

Change-Id: Ic1bb753dd43233cbc6c55368d36bb17dfc24eb75
---
M checkbox.php
M multi.php
M slider.php
M wigo3.body.php
M wigo3.php
5 files changed, 78 insertions(+), 88 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Wigo3 
refs/changes/07/359107/1

diff --git a/checkbox.php b/checkbox.php
index 94f5896..302ec80 100644
--- a/checkbox.php
+++ b/checkbox.php
@@ -40,6 +40,8 @@
   
   #avoid conflicts - checkbox will add check prefix
   $voteid = "set" . $voteid;
+  $htmlVoteId = htmlspecialchars( $voteid );
+  $jsVoteId = htmlspecialchars( Xml::encodeJsVar( $voteid ) );
   
   $set = $args['set'];
   if (!$set)
@@ -69,15 +71,18 @@
if ( count($parts) >= 2 ) {
$title = $parts[0];
$id = $parts[1];
-   $id = str_replace(' ','_',$id);
+   $id = Sanitizer::escapeClass(str_replace(' ','_',$id));
} else {
//someone forgot the id, generate one for them
$title = $parts[0];
$id = Sanitizer::escapeClass($parts[0]);
}
-   $ids[] = "'check{$voteid}-check-{$id}'";
-   $output .= "{$title}";
-   $jshacka[] = "'check{$voteid}-check-{$id}' : 
document.getElementById('checkbox-input-check{$voteid}-check-{$id}').checked?1:0";
+   $jsId = Xml::encodeJsVar( "check{$voteid}-check-{$id}" );
+   $ids[] = $jsId;
+   $htmlId = htmlspecialchars( "check{$voteid}-check-{$id}" );
+   $htmlTitle = htmlspecialchars( $title );
+   $output .= "{$htmlTitle}";
+   $jshacka[] = "$jsId : document.getElementById(" . 
Xml::encodeJsVar( "checkbox-input-check{$voteid}-check-{$id}" ) . 
").checked?1:0";
}
$jshack = "{" . implode(',',$jshacka) . "}";
$output .= "";
@@ -165,16 +170,19 @@
   wfLoadExtensionMessages('wigo3');
   $totalvotes = 
wfMsgExt('wigovotestotal',array('parsemag'),array($countvotes));
 
+  $jsVoteId = Xml::encodeJsVar( $voteid );
+  $htmlVoteId = htmlspecialchars( $voteid );
+
   # script to get my vote
   if (!$bulkmode) {
   $myvotescript = "" .
-  "sajax_do_call('wigogetmyvotes',['{$voteid}'],function (req) 
{" .
+  "sajax_do_call('wigogetmyvotes',[$jsVoteId],function (req) 
{" .
   "if (req.readyState == 4) if (req.status == 200)" .
   "{".
"var res = eval('(' + req.responseText + ')');" .
-   "if ( res['{$voteid}']!== false ) {" .
-   "c = 
document.getElementById(\"checkbox-input-{$voteid}\");" .
- "c.checked=(res['$voteid'] == 1);" .
+   "if ( res[$jsVoteId]!== false ) {" .
+   "c = 
document.getElementById(\"checkbox-input-\" + $jsVoteId);" .
+ "c.checked=(res[$jsVoteId] == 1);" .
"}" .
   "}" .
   "});"  . "";
@@ -182,13 +190,13 @@

   wfLoadExtensionMessages('slider');
   if (array_key_exists('closed',$args) && strcasecmp($args['closed'],"yes") 
=== 0) {
-   return " " .
-   "$output ({$votes})" .
+   return " " .
+   "$output ({$votes})" .
 $myvotescript;
   } else {
-   return " " .
-  "$output ({$votes})" .
-  ($bulkmode ? "" : "  " . wfMsg("slider-votebutton") . 
"") .
+   return " " .
+  "$output ({$votes})" .
+  ($bulkmode ? "" : "  " . wfMsg("slider-votebutton") . "") .
  $myvotescript;
   }
 }
diff --git a/multi.php b/multi.php
index 5005c38..f1160c2 100644
--- a/multi.php
+++ b/multi.php
@@ -135,26 +135,25 @@
   $results[$i] = "0";
 }
 $res->free();
-#format my vote - doesn't work with caching
-/*
-if ($myvote !== null && $myvote === $i) {
-  $line = "{$line}";
-  $resultstr[$i] = "" . $results[$i] . "";
-} else {*/
-  $line = "{$line}";
-  $resultstr[$i] = "" . $results[$i] . 
"";
-/*}*/
+   #format my vote - doesn't work with caching
+   
+   $jsVoteId = Xml::encodeJsVar( $voteid );
+   $htmlVoteId = htmlspecialchars( $voteid );
+   $htmlJsVoteId = htmlspecialchars( $jsVoteId );
+
+   $line = "{$line}";
+   $resultstr[$i] = "" . 
$results[$i] . "";
 $outputlines[] = $parser->recursiveTagParse($line);
   }
 
   # script to get my vote and format it
   $boldscript = "" .
-  "sajax_do_call('multigetmyvote',

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Add contributor list

2017-06-12 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/358508 )

Change subject: Add contributor list
..

Add contributor list

Change-Id: I8a5548d760dc0be7ae6142538c9e94e777fa9709
---
A CREDITS
1 file changed, 3 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/08/358508/1

diff --git a/CREDITS b/CREDITS
new file mode 100644
index 000..094293a
--- /dev/null
+++ b/CREDITS
@@ -0,0 +1,3 @@
+- Tim Starling: main developer
+- C. Scott Ananian: many ideas and some code were taken from his Balancer
+  library.

-- 
To view, visit https://gerrit.wikimedia.org/r/358508
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8a5548d760dc0be7ae6142538c9e94e777fa9709
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/libs/RemexHtml
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix SqlBagOStuff exptime uniqueness assumption

2017-06-10 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/358166 )

Change subject: Fix SqlBagOStuff exptime uniqueness assumption
..

Fix SqlBagOStuff exptime uniqueness assumption

The WMF parser cache tables have some 3 rows with an identical, old
expiry time. So each time deleteObjectsExpiringBefore() is run, the
first batch on each table is correct, but the keys for the second batch
are selected with exptime > X, where X is shared exptime of the many old
rows.

So, use exptime >= X instead.

Change-Id: I6853f64f88f65d4529be2a42c9ed70cfa62cf653
---
M includes/objectcache/SqlBagOStuff.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/66/358166/1

diff --git a/includes/objectcache/SqlBagOStuff.php 
b/includes/objectcache/SqlBagOStuff.php
index a4a6ba8..6c10301 100644
--- a/includes/objectcache/SqlBagOStuff.php
+++ b/includes/objectcache/SqlBagOStuff.php
@@ -588,7 +588,7 @@
while ( true ) {
$conds = $baseConds;
if ( $maxExpTime !== false ) {
-   $conds[] = 'exptime > ' 
. $db->addQuotes( $maxExpTime );
+   $conds[] = 'exptime >= 
' . $db->addQuotes( $maxExpTime );
}
$rows = $db->select(

$this->getTableNameByShard( $i ),

-- 
To view, visit https://gerrit.wikimedia.org/r/358166
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6853f64f88f65d4529be2a42c9ed70cfa62cf653
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Improve test coverage in includes/password

2017-06-05 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/357338 )

Change subject: Improve test coverage in includes/password
..

Improve test coverage in includes/password

From 21% to 82%.

* Added missing @covers, broadened @covers where appropriate.
* Added tests for some code that lacked them.
* Added a parameter to control the use of hash_pbkdf2() so that the pure
  PHP fallback could be tested. In the non-fallback test, force the use
  of the extension, and mark it skipped if it is not installed.

Change-Id: I987e1a89ec343907f4ead7f6192b2d4deb58ac16
---
M includes/password/Pbkdf2Password.php
M tests/phpunit/includes/password/BcryptPasswordTest.php
A tests/phpunit/includes/password/EncryptedPasswordTest.php
M tests/phpunit/includes/password/LayeredParameterizedPasswordTest.php
A tests/phpunit/includes/password/MWOldPasswordTest.php
A tests/phpunit/includes/password/MWSaltedPasswordTest.php
M tests/phpunit/includes/password/PasswordPolicyChecksTest.php
M tests/phpunit/includes/password/PasswordTest.php
M tests/phpunit/includes/password/PasswordTestCase.php
A tests/phpunit/includes/password/Pbkdf2PasswordFallbackTest.php
M tests/phpunit/includes/password/Pbkdf2PasswordTest.php
M tests/phpunit/includes/password/UserPasswordPolicyTest.php
12 files changed, 228 insertions(+), 14 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/38/357338/1

diff --git a/includes/password/Pbkdf2Password.php 
b/includes/password/Pbkdf2Password.php
index 6ffada3..4a8831e 100644
--- a/includes/password/Pbkdf2Password.php
+++ b/includes/password/Pbkdf2Password.php
@@ -41,12 +41,17 @@
return ':';
}
 
+   protected function shouldUseHashExtension() {
+   return isset( $this->config['use-hash-extension'] ) ?
+   $this->config['use-hash-extension'] : function_exists( 
'hash_pbkdf2' );
+   }
+
public function crypt( $password ) {
if ( count( $this->args ) == 0 ) {
$this->args[] = base64_encode( MWCryptRand::generate( 
16, true ) );
}
 
-   if ( function_exists( 'hash_pbkdf2' ) ) {
+   if ( $this->shouldUseHashExtension() ) {
$hash = hash_pbkdf2(
$this->params['algo'],
$password,
diff --git a/tests/phpunit/includes/password/BcryptPasswordTest.php 
b/tests/phpunit/includes/password/BcryptPasswordTest.php
index 8f80362..9b8e01e 100644
--- a/tests/phpunit/includes/password/BcryptPasswordTest.php
+++ b/tests/phpunit/includes/password/BcryptPasswordTest.php
@@ -2,6 +2,10 @@
 
 /**
  * @group large
+ * @covers BcryptPassword
+ * @covers ParameterizedPassword
+ * @covers Password
+ * @covers PasswordFactory
  */
 class BcryptPasswordTest extends PasswordTestCase {
protected function getTypeConfigs() {
diff --git a/tests/phpunit/includes/password/EncryptedPasswordTest.php 
b/tests/phpunit/includes/password/EncryptedPasswordTest.php
new file mode 100644
index 000..0c85653
--- /dev/null
+++ b/tests/phpunit/includes/password/EncryptedPasswordTest.php
@@ -0,0 +1,81 @@
+ [
+   'class' => 'EncryptedPassword',
+   'underlying' => 'pbkdf2',
+   'secrets' => [
+   md5( 'secret1' ),
+   md5( 'secret2' ),
+   ],
+   'cipher' => 'aes-256-cbc',
+   ],
+   'secret1' => [
+   'class' => 'EncryptedPassword',
+   'underlying' => 'pbkdf2',
+   'secrets' => [
+   md5( 'secret1' ),
+   ],
+   'cipher' => 'aes-256-cbc',
+   ],
+   'secret2' => [
+   'class' => 'EncryptedPassword',
+   'underlying' => 'pbkdf2',
+   'secrets' => [
+   md5( 'secret2' ),
+   ],
+   'cipher' => 'aes-256-cbc',
+   ],
+   'pbkdf2' => [
+   'class' => 'Pbkdf2Password',
+   'algo' => 'sha256',
+   'cost' => '10',
+   'length' => '64',
+   ],
+   ];
+   }
+
+   public static function providePasswordTests() {
+   return [
+   // Encrypted with secret1
+   [ true, 
':both:aes-256-cbc:0:izBpxujqC1YbzpCB3qAzgg==:ZqHnitT1pL4YJqKqFES2KEevZYSy2L

[MediaWiki-commits] [Gerrit] operations/puppet[production]: For HHVM set LANG=C.UTF-8

2017-05-10 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/353228 )

Change subject: For HHVM set LANG=C.UTF-8
..

For HHVM set LANG=C.UTF-8

If we set the locale to this using setlocale() in Setup.php, as proposed
in I02943803d26d5b1b3ac, there will be a per-request overhead since HHVM
has a thread-safe setlocale(), so it will need to switch from
en-US.UTF-8 to C.UTF-8 on every request. Better to start up in C.UTF-8,
then the setlocale() call will be faster.

Bug: T107128
Change-Id: Ia661100ef0c5a0ab0365beee6456e3dd8265e977
---
M modules/hhvm/templates/initscripts/hhvm.systemd_override.erb
1 file changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/28/353228/1

diff --git a/modules/hhvm/templates/initscripts/hhvm.systemd_override.erb 
b/modules/hhvm/templates/initscripts/hhvm.systemd_override.erb
index 3293a96..89f185f 100644
--- a/modules/hhvm/templates/initscripts/hhvm.systemd_override.erb
+++ b/modules/hhvm/templates/initscripts/hhvm.systemd_override.erb
@@ -6,6 +6,10 @@
 <%- if !@malloc_arenas.nil? && @malloc_arenas != ''  -%>
 Environment="MALLOC_CONF=narenas:<%= @malloc_arenas %>"
 <%- end -%>
+
+# For T107128
+Environment=LANG=C.UTF-8
+
 EnvironmentFile=
 EnvironmentFile=/etc/default/hhvm
 ExecStart=

-- 
To view, visit https://gerrit.wikimedia.org/r/353228
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia661100ef0c5a0ab0365beee6456e3dd8265e977
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...AbuseFilter[master]: Fix usage of $db->nextSequenceValue()

2017-05-10 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/353198 )

Change subject: Fix usage of $db->nextSequenceValue()
..

Fix usage of $db->nextSequenceValue()

The return value from the method is only suitable for passing to
$db->insert(). To get the inserted ID, you need to call $db->insertId()
even if $db->nextSequenceValue() returned non-null.

Change-Id: Id5a0df17c77445e9f29564a55fb850c3ecad2630
---
M includes/AbuseFilter.class.php
1 file changed, 1 insertion(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/AbuseFilter 
refs/changes/98/353198/1

diff --git a/includes/AbuseFilter.class.php b/includes/AbuseFilter.class.php
index 5ce7337..c531e96 100644
--- a/includes/AbuseFilter.class.php
+++ b/includes/AbuseFilter.class.php
@@ -1118,10 +1118,7 @@
$data['afl_var_dump'] = $var_dump;
$data['afl_id'] = $dbw->nextSequenceValue( 
'abuse_filter_log_afl_id_seq' );
$dbw->insert( 'abuse_filter_log', $data, __METHOD__ );
-   $local_log_ids[] = $dbw->insertId();
-   if ( $data['afl_id'] === null ) {
-   $data['afl_id'] = $dbw->insertId();
-   }
+   $local_log_ids[] = $data['afl_id'] = $dbw->insertId();
// Give grep a chance to find the usages:
// logentry-abusefilter-hit
$entry = new ManualLogEntry( 'abusefilter', 'hit' );

-- 
To view, visit https://gerrit.wikimedia.org/r/353198
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id5a0df17c77445e9f29564a55fb850c3ecad2630
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/AbuseFilter
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Remove EtcdConfig from beta cluster for now

2017-05-03 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351767 )

Change subject: Remove EtcdConfig from beta cluster for now
..

Remove EtcdConfig from beta cluster for now

To match production. We're not planning on immediately deploying it to
production anymore.

Change-Id: Ifc7b1ea802d047ecaec723f71dfc0d26e738f853
---
M wmf-config/CommonSettings.php
D wmf-config/etcd.php
2 files changed, 3 insertions(+), 51 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/67/351767/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 54132e6..2e997cd 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -99,13 +99,9 @@
 # Shorthand when we have no master-slave situation to keep into account
 $wmfLocalServices = $wmfAllServices[$wmfDatacenter];
 
-# Labs-only for testing, eventually etcd.php will be used in production as well
-if ( $wmfRealm === 'labs' ) {
-   # Get configuration from etcd. This gives us the correct 
$wmfMasterDatacenter
-   require "$wmfConfigDir/etcd.php";
-} else {
-   $wmfMasterDatacenter = 'eqiad';
-}
+# Master datacenter
+# The datacenter from which we serve traffic.
+$wmfMasterDatacenter = 'eqiad';
 
 $wmfMasterServices = $wmfAllServices[$wmfMasterDatacenter];
 
diff --git a/wmf-config/etcd.php b/wmf-config/etcd.php
deleted file mode 100644
index 161be4e..000
--- a/wmf-config/etcd.php
+++ /dev/null
@@ -1,44 +0,0 @@
- $wmfLocalServices['etcd'],
-   'protocol' => 'https',
-   'directory' => 
"conftool/v1/mediawiki-config/$wmfDatacenter",
-   'cache' => $localCache,
-   ] ),
-   new EtcdConfig( [
-   'host' => $wmfLocalServices['etcd'],
-   'protocol' => 'https',
-   'directory' => "conftool/v1/mediawiki-config/common",
-   'cache' => $localCache,
-   ] ),
-   ] );
-
-   # Read only mode
-   $wgReadOnly = $etcdConfig->get( 'ReadOnly' );
-
-   # Master datacenter
-   # The datacenter from which we serve traffic.
-   $wmfMasterDatacenter = $etcdConfig->get( 'WMFMasterDatacenter' );
-}
-
-wmfSetupEtcd();

-- 
To view, visit https://gerrit.wikimedia.org/r/351767
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifc7b1ea802d047ecaec723f71dfc0d26e738f853
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: EtcdConfig: Fix infinite timeout bug, and reduce timeout

2017-05-03 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351759 )

Change subject: EtcdConfig: Fix infinite timeout bug, and reduce timeout
..

EtcdConfig: Fix infinite timeout bug, and reduce timeout

removeServer() returns the modified array, rather than passing by
reference, so you have to use the return value to avoid an infinite loop
when a server is down.

Tune the timeout downwards, to 2s. With three servers in the SRV pool,
if they are all unreachable, this will mean an overall request time of
6s, which is conveniently less than the APC lock time and the cache
TTL (9-10s). If the APC lock time is significantly shorter than the time
it takes to do the HTTP requests, then additional threads join in
waiting for the server. This could have stability consequences if the
maximum HHVM worker count is exceeded.

Change-Id: I3176aa41b8833c0ba0b668859e59911cd4392250
---
M includes/config/EtcdConfig.php
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/59/351759/1

diff --git a/includes/config/EtcdConfig.php b/includes/config/EtcdConfig.php
index fd5c3f7..880cf9f 100644
--- a/includes/config/EtcdConfig.php
+++ b/includes/config/EtcdConfig.php
@@ -73,7 +73,7 @@
'encoding' => 'JSON',
'cacheTTL' => 10,
'skewTTL' => 1,
-   'timeout' => 10
+   'timeout' => 2
];
 
$this->host = $params['host'];
@@ -215,7 +215,7 @@
}
 
// Avoid the server next time if that failed
-   $dsd->removeServer( $server, $servers );
+   $servers = $dsd->removeServer( $server, $servers );
} while ( $servers );
 
return [ $config, $error, $retry ];

-- 
To view, visit https://gerrit.wikimedia.org/r/351759
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3176aa41b8833c0ba0b668859e59911cd4392250
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Revert "Enable EtcdConfig in production"

2017-05-02 Thread Tim Starling (Code Review)
Hello Giuseppe Lavagetto, Krinkle, Brion VIBBER, jenkins-bot, Volans,

I'd like you to do a code review.  Please visit

https://gerrit.wikimedia.org/r/351550

to review the following change.


Change subject: Revert "Enable EtcdConfig in production"
..

Revert "Enable EtcdConfig in production"

This reverts commit 713f6474acf63d2524cdd131ffb849e72ca2fe62.

Change-Id: I795cbd60ef3dc5c5bd5ec961f7674be2d2db7826
---
M wmf-config/CommonSettings.php
1 file changed, 7 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/50/351550/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 12cbab9..86ddc58 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -99,8 +99,13 @@
 # Shorthand when we have no master-slave situation to keep into account
 $wmfLocalServices = $wmfAllServices[$wmfDatacenter];
 
-# Get configuration from etcd. This gives us the correct $wmfMasterDatacenter
-require "$wmfConfigDir/etcd.php";
+# Labs-only for testing, eventually etcd.php will be used in production as well
+if ( $wmfRealm === 'labs' ) {
+   # Get configuration from etcd. This gives us the correct 
$wmfMasterDatacenter
+   require "$wmfConfigDir/etcd.php";
+} else {
+   $wmfMasterDatacenter = 'codfw';
+}
 
 $wmfMasterServices = $wmfAllServices[$wmfMasterDatacenter];
 

-- 
To view, visit https://gerrit.wikimedia.org/r/351550
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I795cbd60ef3dc5c5bd5ec961f7674be2d2db7826
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Brion VIBBER 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Krinkle 
Gerrit-Reviewer: Volans 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Revert "wmf-config: readonly is set in etcd now"

2017-05-02 Thread Tim Starling (Code Review)
Hello Giuseppe Lavagetto, jenkins-bot, Volans,

I'd like you to do a code review.  Please visit

https://gerrit.wikimedia.org/r/351551

to review the following change.


Change subject: Revert "wmf-config: readonly is set in etcd now"
..

Revert "wmf-config: readonly is set in etcd now"

This reverts commit 566b1fac6655c5e140dac7377b5bf7d7b05398dc.

Change-Id: I7d3a7634f6c6d5f60ffd94c3bfac98d51fa9e92a
---
M wmf-config/db-codfw.php
M wmf-config/db-eqiad.php
2 files changed, 11 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/51/351551/1

diff --git a/wmf-config/db-codfw.php b/wmf-config/db-codfw.php
index b0545e0..9a1c886 100644
--- a/wmf-config/db-codfw.php
+++ b/wmf-config/db-codfw.php
@@ -638,6 +638,10 @@
 ],
 
 # This key must exist for the master switch script to work.
+#
+# These read only messages should currently be kept,
+# to prevent accidental write to eqiad from codfw,
+# when the master dc is eqiad.
 'readOnlyBySection' => [
 #  's1'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
 #  's2'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
diff --git a/wmf-config/db-eqiad.php b/wmf-config/db-eqiad.php
index 7584146..5ba7b4e 100644
--- a/wmf-config/db-eqiad.php
+++ b/wmf-config/db-eqiad.php
@@ -663,13 +663,13 @@
 
 # This key must exist for the master switch script to work
 'readOnlyBySection' => [
-#  's1'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
-#  's2'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
-#  'DEFAULT' => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes', # s3
-#  's4'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
-#  's5'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
-#  's6'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
-#  's7'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
+   's1'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
+   's2'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
+   'DEFAULT' => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes', # s3
+   's4'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
+   's5'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
+   's6'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
+   's7'  => 'MediaWiki is in read-only mode for maintenance. Please 
try again in a few minutes',
 ],
 
 ];

-- 
To view, visit https://gerrit.wikimedia.org/r/351551
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7d3a7634f6c6d5f60ffd94c3bfac98d51fa9e92a
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Volans 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Add $wmfMasterDatacenter to meta=siteinfo

2017-05-01 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351232 )

Change subject: Add $wmfMasterDatacenter to meta=siteinfo
..

Add $wmfMasterDatacenter to meta=siteinfo

Bug: T156924
Change-Id: Id75b3d3c2ca9b6b0552e5e07d24436f837c58199
---
M wmf-config/CommonSettings.php
1 file changed, 9 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/32/351232/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 2794a57..28cdbc4 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -296,6 +296,15 @@
return true;
 };
 
+// Add some useful config data to query=siteinfo
+$wgHooks['APIQuerySiteInfoGeneralInfo'][] = function ( $module, &$data ) {
+   global $wmfMasterDatacenter;
+   $data['wmfconfig'] = [
+   'wmfMasterDatacenter' => $wmfMasterDatacenter
+   ];
+   return true;
+};
+
 # used for mysql/search settings
 $tmarray = getdate( time() );
 $hour = $tmarray['hours'];

-- 
To view, visit https://gerrit.wikimedia.org/r/351232
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id75b3d3c2ca9b6b0552e5e07d24436f837c58199
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/puppet[production]: Disable suppress_san_warnings

2017-05-01 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/351223 )

Change subject: Disable suppress_san_warnings
..


Disable suppress_san_warnings

conftool::__init__: Could not load driver etcd: 'module' object has no 
attribute 'SubjectAltNameWarning'

Tested by editing config.yaml on puppetmaster1001

Change-Id: Ieef58f2578226b4d4b4ba792b707ef041bc0c1a5
---
M modules/conftool/manifests/config.pp
1 file changed, 3 insertions(+), 1 deletion(-)

Approvals:
  Tim Starling: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/modules/conftool/manifests/config.pp 
b/modules/conftool/manifests/config.pp
index 5a9090a..8b8e424 100644
--- a/modules/conftool/manifests/config.pp
+++ b/modules/conftool/manifests/config.pp
@@ -30,7 +30,9 @@
 tcpircbot_host => $tcpircbot_host,
 tcpircbot_port => $tcpircbot_port,
 driver_options => {
-allow_reconnect => true,
+allow_reconnect   => true,
+# requires update to python-etcd
+suppress_san_warnings => false
 },
 namespace  => $namespace,
 }),

-- 
To view, visit https://gerrit.wikimedia.org/r/351223
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ieef58f2578226b4d4b4ba792b707ef041bc0c1a5
Gerrit-PatchSet: 2
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Tim Starling 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/puppet[production]: Disable suppress_san_warnings

2017-05-01 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351223 )

Change subject: Disable suppress_san_warnings
..

Disable suppress_san_warnings

conftool::__init__: Could not load driver etcd: 'module' object has no 
attribute 'SubjectAltNameWarning'

Tested by editing config.yaml on puppetmaster1001

Change-Id: Ieef58f2578226b4d4b4ba792b707ef041bc0c1a5
---
M modules/conftool/manifests/config.pp
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/23/351223/1

diff --git a/modules/conftool/manifests/config.pp 
b/modules/conftool/manifests/config.pp
index 5a9090a..18465e3 100644
--- a/modules/conftool/manifests/config.pp
+++ b/modules/conftool/manifests/config.pp
@@ -31,6 +31,8 @@
 tcpircbot_port => $tcpircbot_port,
 driver_options => {
 allow_reconnect => true,
+# requires update to python-etcd
+suppress_san_warnings => false
 },
 namespace  => $namespace,
 }),

-- 
To view, visit https://gerrit.wikimedia.org/r/351223
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ieef58f2578226b4d4b4ba792b707ef041bc0c1a5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/puppet[production]: conftool: add mwconfig object type, define the first couple ...

2017-05-01 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/347360 )

Change subject: conftool: add mwconfig object type, define the first couple 
variables
..


conftool: add mwconfig object type, define the first couple variables

Change-Id: I3391f700328cdf02dc33c57f2de95ca25dd8a9ff
---
A conftool-data/mwconfig/data.yaml
M modules/profile/files/conftool/schema.yaml
2 files changed, 14 insertions(+), 0 deletions(-)

Approvals:
  Giuseppe Lavagetto: Looks good to me, approved
  Tim Starling: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/conftool-data/mwconfig/data.yaml b/conftool-data/mwconfig/data.yaml
new file mode 100644
index 000..4b2e559
--- /dev/null
+++ b/conftool-data/mwconfig/data.yaml
@@ -0,0 +1,6 @@
+codfw:
+  - ReadOnly
+eqiad:
+  - ReadOnly
+common:
+  - WMFMasterDatacenter
diff --git a/modules/profile/files/conftool/schema.yaml 
b/modules/profile/files/conftool/schema.yaml
index a22d9c5..34aefdd 100644
--- a/modules/profile/files/conftool/schema.yaml
+++ b/modules/profile/files/conftool/schema.yaml
@@ -15,3 +15,11 @@
 ttl:
   type: "int"
   default: 300
+mwconfig:
+  path: "mediawiki-config"
+  tags:
+- scope
+  schema:
+val:
+  type: "any"
+  default: null

-- 
To view, visit https://gerrit.wikimedia.org/r/347360
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I3391f700328cdf02dc33c57f2de95ca25dd8a9ff
Gerrit-PatchSet: 7
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Giuseppe Lavagetto 
Gerrit-Reviewer: Giuseppe Lavagetto 
Gerrit-Reviewer: Tim Starling 
Gerrit-Reviewer: Volans 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[wmf/1.29.0-wmf.21]: [1.29.0-wmf.21] config: Use less generic cache key, and not ...

2017-05-01 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351217 )

Change subject: [1.29.0-wmf.21] config: Use less generic cache key, and not 
fragmented by wiki
..

[1.29.0-wmf.21] config: Use less generic cache key, and not fragmented by wiki

* Use makeGlobalKey instead of makeKey to avoid fragmenting
  the cache by the wiki-specific key space.

* Add "EtcdConfig" and $this->host to the cache key to make
  it less generic and less likely to conflict with unrelated values
  in case multiple instances exist somewhere in the MediaWiki run
  time (or during testing/debugging).

Adding $this->host should be fine given we were already including
the directory within that host. Although I do recognise that we
may want to encourage slow cache rollover instead of instance
cache rollover in case the host name needs to changed. Perhaps
we should have some kind of canonical name for the instance for
the purpose of caching so that directory or host can be changed
without changing the cache key, and thus have cache roll over
normally over the course of 10 seconds, instead of immediately.

On the other hand, deployment will likely already spread things
out a little bit.

In addition, due to the cache being kept indefinitely this means
the old key will stay indefinitely. Perhaps not an issue given
it's just one key, which will clear on restart eventually?

Bug: T156924
Change-Id: I622e6618f2d4171626e4b272bcb11a97a85cb770
---
M includes/config/EtcdConfig.php
1 file changed, 5 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/17/351217/1

diff --git a/includes/config/EtcdConfig.php b/includes/config/EtcdConfig.php
index d3fbd65..e47fe66 100644
--- a/includes/config/EtcdConfig.php
+++ b/includes/config/EtcdConfig.php
@@ -54,8 +54,6 @@
private $skewCacheTTL;
/** @var integer */
private $timeout;
-   /** @var string */
-   private $directoryHash;
 
/**
 * @param array $params Parameter map:
@@ -81,7 +79,6 @@
$this->host = $params['host'];
$this->protocol = $params['protocol'];
$this->directory = trim( $params['directory'], '/' );
-   $this->directoryHash = sha1( $this->directory );
$this->encoding = $params['encoding'];
$this->skewCacheTTL = $params['skewTTL'];
$this->baseCacheTTL = max( $params['cacheTTL'] - 
$this->skewCacheTTL, 0 );
@@ -128,7 +125,11 @@
}
 
$now = microtime( true );
-   $key = $this->srvCache->makeKey( 'variable', 
$this->directoryHash );
+   $key = $this->srvCache->makeGlobalKey(
+   __CLASS__,
+   $this->host,
+   $this->directory
+   );
 
// Get the cached value or block until it is regenerated (by 
this or another thread)...
$data = null; // latest config info

-- 
To view, visit https://gerrit.wikimedia.org/r/351217
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I622e6618f2d4171626e4b272bcb11a97a85cb770
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: wmf/1.29.0-wmf.21
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Krinkle 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[wmf/1.29.0-wmf.21]: EtcdConfig: use array_key_exists(), the value can be null

2017-05-01 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351208 )

Change subject: EtcdConfig: use array_key_exists(), the value can be null
..

EtcdConfig: use array_key_exists(), the value can be null

Change-Id: Iae175401da8d49f5475d8c88cc5390b16e7cba32
(cherry picked from commit 4e6270223875bc09c816931e2ac7a992d963205e)
---
M includes/config/EtcdConfig.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/08/351208/1

diff --git a/includes/config/EtcdConfig.php b/includes/config/EtcdConfig.php
index d3fbd65..06d8dfb 100644
--- a/includes/config/EtcdConfig.php
+++ b/includes/config/EtcdConfig.php
@@ -251,7 +251,7 @@
 
$name = basename( $node['key'] );
$value = $this->unserialize( $node['value'] );
-   if ( !is_array( $value ) || !isset( $value['val'] ) ) {
+   if ( !is_array( $value ) || !array_key_exists( 'val', 
$value ) ) {
return [ null, "Failed to parse value for 
'$name'.", false ];
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/351208
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iae175401da8d49f5475d8c88cc5390b16e7cba32
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: wmf/1.29.0-wmf.21
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Enable EtcdConfig in production

2017-04-30 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/351132 )

Change subject: Enable EtcdConfig in production
..

Enable EtcdConfig in production

Ideally, Iae175401da8d should be present before deployment, but failing
that, the relevant variables can just be set to false instead of null.

Before this change is deployed, I3391f700328cdf02d should be merged and
deployed, and the etcd keys initialised.

Change-Id: I74b332c6c4af72085bd21479009ff2f2dadbb9eb
---
M wmf-config/CommonSettings.php
1 file changed, 2 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/32/351132/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 2794a57..09ea4b0 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -99,13 +99,8 @@
 # Shorthand when we have no master-slave situation to keep into account
 $wmfLocalServices = $wmfAllServices[$wmfDatacenter];
 
-# Labs-only for testing, eventually etcd.php will be used in production as well
-if ( $wmfRealm === 'labs' ) {
-   # Get configuration from etcd. This gives us the correct 
$wmfMasterDatacenter
-   require "$wmfConfigDir/etcd.php";
-} else {
-   $wmfMasterDatacenter = 'codfw';
-}
+# Get configuration from etcd. This gives us the correct $wmfMasterDatacenter
+require "$wmfConfigDir/etcd.php";
 
 $wmfMasterServices = $wmfAllServices[$wmfMasterDatacenter];
 

-- 
To view, visit https://gerrit.wikimedia.org/r/351132
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I74b332c6c4af72085bd21479009ff2f2dadbb9eb
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: EtcdConfig: use array_key_exists(), the value can be null

2017-04-27 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/350793 )

Change subject: EtcdConfig: use array_key_exists(), the value can be null
..

EtcdConfig: use array_key_exists(), the value can be null

Change-Id: Iae175401da8d49f5475d8c88cc5390b16e7cba32
---
M includes/config/EtcdConfig.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/93/350793/1

diff --git a/includes/config/EtcdConfig.php b/includes/config/EtcdConfig.php
index d3fbd65..06d8dfb 100644
--- a/includes/config/EtcdConfig.php
+++ b/includes/config/EtcdConfig.php
@@ -251,7 +251,7 @@
 
$name = basename( $node['key'] );
$value = $this->unserialize( $node['value'] );
-   if ( !is_array( $value ) || !isset( $value['val'] ) ) {
+   if ( !is_array( $value ) || !array_key_exists( 'val', 
$value ) ) {
return [ null, "Failed to parse value for 
'$name'.", false ];
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/350793
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iae175401da8d49f5475d8c88cc5390b16e7cba32
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Improve HTTP logging

2017-04-27 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/350787 )

Change subject: Improve HTTP logging
..

Improve HTTP logging

* Log HTTP debug lines to the "http" channel instead of wfDebug()
* Add the ability to do debug logging to MultiHttpClient
* Add a static method Http::createMultiClient() which provides a
  MultiHttpClient configured similarly to the way individual requests
  are configured, respecting the wiki's $wgHTTPTimeout and writing debug
  logs.
* In EtcdConfig, pass the logger instance through to the MultiHttpClient
  backend.

Change-Id: Ic5bdcb0cae95d7b3715ab5261758be082751c3ff
---
M includes/config/EtcdConfig.php
M includes/http/Http.php
M includes/libs/MultiHttpClient.php
3 files changed, 60 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/87/350787/1

diff --git a/includes/config/EtcdConfig.php b/includes/config/EtcdConfig.php
index d3fbd65..1a488bd 100644
--- a/includes/config/EtcdConfig.php
+++ b/includes/config/EtcdConfig.php
@@ -23,6 +23,7 @@
 
 use Psr\Log\LoggerAwareInterface;
 use Psr\Log\LoggerInterface;
+use MediaWiki\Logger\LoggerFactory;
 use Wikimedia\WaitConditionLoop;
 
 /**
@@ -98,12 +99,14 @@
$this->logger = new Psr\Log\NullLogger();
$this->http = new MultiHttpClient( [
'connTimeout' => $this->timeout,
-   'reqTimeout' => $this->timeout
+   'reqTimeout' => $this->timeout,
+   'logger' => $this->logger
] );
}
 
public function setLogger( LoggerInterface $logger ) {
$this->logger = $logger;
+   $this->http->setLogger( $logger );
}
 
public function has( $name ) {
diff --git a/includes/http/Http.php b/includes/http/Http.php
index 889cb60..4f21ce2 100644
--- a/includes/http/Http.php
+++ b/includes/http/Http.php
@@ -59,7 +59,8 @@
 * @return string|bool (bool)false on failure or a string on success
 */
public static function request( $method, $url, $options = [], $caller = 
__METHOD__ ) {
-   wfDebug( "HTTP: $method: $url\n" );
+   $logger = LoggerFactory::getInstance( 'http' );
+   $logger->debug( "$method: $url" );
 
$options['method'] = strtoupper( $method );
 
@@ -77,7 +78,6 @@
return $req->getContent();
} else {
$errors = $status->getErrorsByType( 'error' );
-   $logger = LoggerFactory::getInstance( 'http' );
$logger->warning( Status::wrap( $status )->getWikiText( 
false, false, 'en' ),
[ 'error' => $errors, 'caller' => $caller, 
'content' => $req->getContent() ] );
return false;
@@ -164,4 +164,20 @@
 
return "";
}
+
+   /**
+* Get a configured MultiHttpClient
+* @param array $options
+*/
+   public static function createMultiClient( $options = [] ) {
+   global $wgHTTPConnectTimeout, $wgHTTPTimeout, $wgHTTPProxy;
+
+   return new MultiHttpClient( $options + [
+   'connTimeout' => $wgHTTPConnectTimeout,
+   'reqTimeout' => $wgHTTPTimeout,
+   'userAgent' => self::userAgent(),
+   'proxy' => $wgHTTPProxy,
+   'logger' => LoggerFactory::getInstance( 'http' )
+   ] );
+   }
 }
diff --git a/includes/libs/MultiHttpClient.php 
b/includes/libs/MultiHttpClient.php
index f0b44a5..b077059 100644
--- a/includes/libs/MultiHttpClient.php
+++ b/includes/libs/MultiHttpClient.php
@@ -59,6 +59,8 @@
protected $proxy;
/** @var string */
protected $userAgent = 'wikimedia/multi-http-client v1.0';
+   /** @var Psr\Log\LoggerInterface */
+   protected $logger;
 
/**
 * @param array $options
@@ -78,7 +80,8 @@
}
}
static $opts = [
-   'connTimeout', 'reqTimeout', 'usePipelining', 
'maxConnsPerHost', 'proxy', 'userAgent'
+   'connTimeout', 'reqTimeout', 'usePipelining', 
'maxConnsPerHost',
+   'proxy', 'userAgent', 'logger'
];
foreach ( $opts as $key ) {
if ( isset( $options[$key] ) ) {
@@ -162,6 +165,7 @@
} elseif ( !isset( $req['url'] ) ) {
throw new Exception( "Request has no 'url' 
field set." );
}
+   $this->logDebug( "{$req['method']}: {$req['url']}" );
$req['query'] = isset( $req['query'] ) ? $req['query'] 
: [];
$headers = []; // normalized headers
  

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Make "eval.php -d 2" work again

2017-04-27 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/350786 )

Change subject: Make "eval.php -d 2" work again
..

Make "eval.php -d 2" work again

Instead of iterating through load balancer info, just set the flag in
the master and replica connections, since that seems more robust.

Deprecate LoadBalancer::setServerInfo() and
LoadBalancer::getServerInfo(), no remaining callers in core or
in-tree extensions, I think I added the function just for this feature.

Do a service reset, since Logger instances are injected into
LoadBalancer by ServiceWiring (via MWLBFactory::applyDefaultConfig()).
Similarly CryptRand, MimeAnalyzer.

Fix the usage text for shell.php

Change-Id: I3e85a6e8cfa1243a0371cfb3ce1c18665e8c711d
---
M RELEASE-NOTES-1.30
M includes/libs/rdbms/loadbalancer/ILoadBalancer.php
M includes/libs/rdbms/loadbalancer/LoadBalancer.php
M maintenance/eval.php
M maintenance/shell.php
5 files changed, 26 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/86/350786/1

diff --git a/RELEASE-NOTES-1.30 b/RELEASE-NOTES-1.30
index 3c1ae8d..4cfa19b 100644
--- a/RELEASE-NOTES-1.30
+++ b/RELEASE-NOTES-1.30
@@ -39,7 +39,8 @@
 * …
 
 === Other changes in 1.30 ===
-* …
+* LoadBalancer::getServerInfo() and LoadBalancer::setServerInfo() are
+  deprecated. There are no known callers.
 
 == Compatibility ==
 MediaWiki 1.30 requires PHP 5.5.9 or later. There is experimental support for
diff --git a/includes/libs/rdbms/loadbalancer/ILoadBalancer.php 
b/includes/libs/rdbms/loadbalancer/ILoadBalancer.php
index ceb8d07..79827a2 100644
--- a/includes/libs/rdbms/loadbalancer/ILoadBalancer.php
+++ b/includes/libs/rdbms/loadbalancer/ILoadBalancer.php
@@ -286,6 +286,8 @@
 * Return the server info structure for a given index, or false if the 
index is invalid.
 * @param int $i
 * @return array|bool
+*
+* @deprecated Since 1.30, no alternative
 */
public function getServerInfo( $i );
 
@@ -294,6 +296,8 @@
 * is created if it doesn't exist
 * @param int $i
 * @param array $serverInfo
+*
+* @deprecated Since 1.30, construct new object
 */
public function setServerInfo( $i, array $serverInfo );
 
diff --git a/includes/libs/rdbms/loadbalancer/LoadBalancer.php 
b/includes/libs/rdbms/loadbalancer/LoadBalancer.php
index e2fbf72..0fc00a8 100644
--- a/includes/libs/rdbms/loadbalancer/LoadBalancer.php
+++ b/includes/libs/rdbms/loadbalancer/LoadBalancer.php
@@ -1033,7 +1033,11 @@
return ( $name != '' ) ? $name : 'localhost';
}
 
+   /**
+* @deprecated Since 1.30, no alternative
+*/
public function getServerInfo( $i ) {
+   wfDeprecated( __METHOD__, '1.30' );
if ( isset( $this->mServers[$i] ) ) {
return $this->mServers[$i];
} else {
@@ -1041,7 +1045,11 @@
}
}
 
+   /**
+* @deprecated Since 1.30, construct new object
+*/
public function setServerInfo( $i, array $serverInfo ) {
+   wfDeprecated( __METHOD__, '1.30' );
$this->mServers[$i] = $serverInfo;
}
 
diff --git a/maintenance/eval.php b/maintenance/eval.php
index ee8bdd6..40d29ef 100644
--- a/maintenance/eval.php
+++ b/maintenance/eval.php
@@ -32,6 +32,7 @@
 
 use MediaWiki\Logger\LoggerFactory;
 use MediaWiki\Logger\ConsoleSpi;
+use MediaWiki\MediaWikiServices;
 
 $optionsWithArgs = [ 'd' ];
 
@@ -41,15 +42,12 @@
$d = $options['d'];
if ( $d > 0 ) {
LoggerFactory::registerProvider( new ConsoleSpi );
+   // Some services hold Logger instances in object properties
+   MediaWikiServices::resetGlobalInstance();
}
if ( $d > 1 ) {
-   $lb = wfGetLB();
-   $serverCount = $lb->getServerCount();
-   for ( $i = 0; $i < $serverCount; $i++ ) {
-   $server = $lb->getServerInfo( $i );
-   $server['flags'] |= DBO_DEBUG;
-   $lb->setServerInfo( $i, $server );
-   }
+   wfGetDB( DB_MASTER )->setFlag( DBO_DEBUG );
+   wfGetDB( DB_REPLICA )->setFlag( DBO_DEBUG );
}
 }
 
diff --git a/maintenance/shell.php b/maintenance/shell.php
index 9f2306f..65c353a 100644
--- a/maintenance/shell.php
+++ b/maintenance/shell.php
@@ -36,6 +36,7 @@
 
 use MediaWiki\Logger\LoggerFactory;
 use MediaWiki\Logger\ConsoleSpi;
+use MediaWiki\MediaWikiServices;
 
 require_once __DIR__ . '/Maintenance.php';
 
@@ -49,8 +50,8 @@
parent::__construct();
$this->addOption( 'd',
'For back compatibility with eval.php. ' .
-   '0 send debug to stderr. ' .
-   'With 1 additionally i

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Add ConsoleLogger, use it for eval.php -d

2017-04-27 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/350781 )

Change subject: Add ConsoleLogger, use it for eval.php -d
..

Add ConsoleLogger, use it for eval.php -d

eval.php previously set $wgDebugLogFile to /dev/stdout. This had the
following problems:

* It doesn't work if the maintenance script is executed via sudo, since
  /dev/stdout is typically owned by the original user, so MW can't open
  it. Using php://stdout worked on HHVM but not PHP.
* Setting $wgDebugLogFile has no effect if the wiki uses MonologSpi.
* Setting $wgDebugLogFile has no effect on channels configured with
  $wgDebugLogGroups.
* stderr is a more appropriate place to send logging output.
* Writing to configuration variables is discouraged.

So, add ConsoleSpi, which is a very simple logging service provider
which sends all messages to stderr. This should be suitable for
debugging with eval.php or shell.php in WMF production or beta.

Change-Id: Ib0d6ce45e0cbecd58263fc4e360c63d4149acb3a
---
M autoload.php
A includes/debug/logger/ConsoleLogger.php
A includes/debug/logger/ConsoleSpi.php
M maintenance/eval.php
M maintenance/shell.php
5 files changed, 43 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/81/350781/1

diff --git a/autoload.php b/autoload.php
index f609ffc..1141c39 100644
--- a/autoload.php
+++ b/autoload.php
@@ -876,6 +876,8 @@
'MediaWiki\\Linker\\LinkRenderer' => __DIR__ . 
'/includes/linker/LinkRenderer.php',
'MediaWiki\\Linker\\LinkRendererFactory' => __DIR__ . 
'/includes/linker/LinkRendererFactory.php',
'MediaWiki\\Linker\\LinkTarget' => __DIR__ . 
'/includes/linker/LinkTarget.php',
+   'MediaWiki\\Logger\\ConsoleLogger' => __DIR__ . 
'/includes/debug/logger/ConsoleLogger.php',
+   'MediaWiki\\Logger\\ConsoleSpi' => __DIR__ . 
'/includes/debug/logger/ConsoleSpi.php',
'MediaWiki\\Logger\\LegacyLogger' => __DIR__ . 
'/includes/debug/logger/LegacyLogger.php',
'MediaWiki\\Logger\\LegacySpi' => __DIR__ . 
'/includes/debug/logger/LegacySpi.php',
'MediaWiki\\Logger\\LoggerFactory' => __DIR__ . 
'/includes/debug/logger/LoggerFactory.php',
diff --git a/includes/debug/logger/ConsoleLogger.php 
b/includes/debug/logger/ConsoleLogger.php
new file mode 100644
index 000..5a5e507
--- /dev/null
+++ b/includes/debug/logger/ConsoleLogger.php
@@ -0,0 +1,21 @@
+channel = $channel;
+   }
+
+   public function log( $level, $message, array $context = [] ) {
+   fwrite( STDERR, "[$level] " .
+   LegacyLogger::format( $this->channel, $message, 
$context ) );
+   }
+}
diff --git a/includes/debug/logger/ConsoleSpi.php 
b/includes/debug/logger/ConsoleSpi.php
new file mode 100644
index 000..e29b98d
--- /dev/null
+++ b/includes/debug/logger/ConsoleSpi.php
@@ -0,0 +1,11 @@
+ 0 ) {
-   $wgDebugLogFile = '/dev/stdout';
+   LoggerFactory::registerProvider( new ConsoleSpi );
}
if ( $d > 1 ) {
$lb = wfGetLB();
diff --git a/maintenance/shell.php b/maintenance/shell.php
index 47ef804..9f2306f 100644
--- a/maintenance/shell.php
+++ b/maintenance/shell.php
@@ -34,6 +34,9 @@
  * @author Gergő Tisza 
  */
 
+use MediaWiki\Logger\LoggerFactory;
+use MediaWiki\Logger\ConsoleSpi;
+
 require_once __DIR__ . '/Maintenance.php';
 
 /**
@@ -46,7 +49,7 @@
parent::__construct();
$this->addOption( 'd',
'For back compatibility with eval.php. ' .
-   '0 send debug to stdout. ' .
+   '0 send debug to stderr. ' .
'With 1 additionally initialize database with debugging 
',
false, true
);
@@ -77,11 +80,9 @@
 * For back compatibility with eval.php
 */
protected function setupLegacy() {
-   global $wgDebugLogFile;
-
$d = intval( $this->getOption( 'd' ) );
if ( $d > 0 ) {
-   $wgDebugLogFile = 'php://stdout';
+   LoggerFactory::registerProvider( new ConsoleSpi );
}
if ( $d > 1 ) {
# Set DBO_DEBUG (equivalent of $wgDebugDumpSql)

-- 
To view, visit https://gerrit.wikimedia.org/r/350781
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib0d6ce45e0cbecd58263fc4e360c63d4149acb3a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Document rationale or primary use case for many indexes

2017-04-23 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/349872 )

Change subject: Document rationale or primary use case for many indexes
..

Document rationale or primary use case for many indexes

Change-Id: I1f0a13adbc6f618f38c3b05a1d356d76e2f5bd23
---
M maintenance/tables.sql
1 file changed, 109 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/72/349872/1

diff --git a/maintenance/tables.sql b/maintenance/tables.sql
index 38fef45..79249b0 100644
--- a/maintenance/tables.sql
+++ b/maintenance/tables.sql
@@ -311,9 +311,20 @@
   page_lang varbinary(35) DEFAULT NULL
 ) /*$wgDBTableOptions*/;
 
+-- The title index. Care must be taken to always specify a namespace when
+-- by title, so that the index is used. Even listing all known namespaces
+-- with IN() is better than omitting page_namespace from the WHERE clause.
 CREATE UNIQUE INDEX /*i*/name_title ON /*_*/page (page_namespace,page_title);
+
+-- The index for Special:Random
 CREATE INDEX /*i*/page_random ON /*_*/page (page_random);
+
+-- @fixme Not used for anything? ApiQueryAllPages has an unconditional
+-- namespace filter so should be using the Special:Shortpages index when
+-- minsize/maxsize are specified.
 CREATE INDEX /*i*/page_len ON /*_*/page (page_len);
+
+-- The index for Special:Shortpages and Special:Longpages
 CREATE INDEX /*i*/page_redirect_namespace_len ON /*_*/page (page_is_redirect, 
page_namespace, page_len);
 
 --
@@ -375,11 +386,26 @@
 ) /*$wgDBTableOptions*/ MAX_ROWS=1000 AVG_ROW_LENGTH=1024;
 -- In case tables are created as MyISAM, use row hints for MySQL <5.0 to avoid 
4GB limit
 
+-- The index is proposed for removal, do not use it in new code: T163532.
+-- Used for ordering revisions within a page by rev_id, which is usually
+-- incorrect, since rev_timestamp is normally the correct order. It can also
+-- be used by dumpBackup.php, if a page and rev_id range is specified.
 CREATE INDEX /*i*/rev_page_id ON /*_*/revision (rev_page, rev_id);
+
+-- Used by ApiQueryAllRevisions
 CREATE INDEX /*i*/rev_timestamp ON /*_*/revision (rev_timestamp);
+
+-- History index
 CREATE INDEX /*i*/page_timestamp ON /*_*/revision (rev_page,rev_timestamp);
+
+-- Logged-in user contributions index
 CREATE INDEX /*i*/user_timestamp ON /*_*/revision (rev_user,rev_timestamp);
+
+-- Anonymous user countributions index
 CREATE INDEX /*i*/usertext_timestamp ON /*_*/revision 
(rev_user_text,rev_timestamp);
+
+-- Credits index. This is scanned in order to compile credits lists for pages,
+-- in ApiQueryContributors. 
 CREATE INDEX /*i*/page_user_timestamp ON /*_*/revision 
(rev_page,rev_user,rev_timestamp);
 
 --
@@ -499,8 +525,14 @@
   ar_content_format varbinary(64) DEFAULT NULL
 ) /*$wgDBTableOptions*/;
 
+-- Index for Special:Undelete to page through deleted revisions
 CREATE INDEX /*i*/name_title_timestamp ON /*_*/archive 
(ar_namespace,ar_title,ar_timestamp);
+
+-- Index for Special:DeletedContributions
 CREATE INDEX /*i*/ar_usertext_timestamp ON /*_*/archive 
(ar_user_text,ar_timestamp);
+
+-- Index for linking archive rows with tables that normally link with revision
+-- rows, such as change_tag.
 CREATE INDEX /*i*/ar_revid ON /*_*/archive (ar_rev_id);
 
 
@@ -521,8 +553,13 @@
   pl_title varchar(255) binary NOT NULL default ''
 ) /*$wgDBTableOptions*/;
 
+-- Forward index, for page edit, save
 CREATE UNIQUE INDEX /*i*/pl_from ON /*_*/pagelinks 
(pl_from,pl_namespace,pl_title);
+
+-- Reverse index, for Special:Whatlinkshere
 CREATE INDEX /*i*/pl_namespace ON /*_*/pagelinks 
(pl_namespace,pl_title,pl_from);
+
+-- Index for Special:Whatlinkshere with namespace filter
 CREATE INDEX /*i*/pl_backlinks_namespace ON /*_*/pagelinks 
(pl_from_namespace,pl_namespace,pl_title,pl_from);
 
 
@@ -543,8 +580,13 @@
   tl_title varchar(255) binary NOT NULL default ''
 ) /*$wgDBTableOptions*/;
 
+-- Forward index, for page edit, save
 CREATE UNIQUE INDEX /*i*/tl_from ON /*_*/templatelinks 
(tl_from,tl_namespace,tl_title);
+
+-- Reverse index, for Special:Whatlinkshere
 CREATE INDEX /*i*/tl_namespace ON /*_*/templatelinks 
(tl_namespace,tl_title,tl_from);
+
+-- Index for Special:Whatlinkshere with namespace filter
 CREATE INDEX /*i*/tl_backlinks_namespace ON /*_*/templatelinks 
(tl_from_namespace,tl_namespace,tl_title,tl_from);
 
 
@@ -565,8 +607,13 @@
   il_to varchar(255) binary NOT NULL default ''
 ) /*$wgDBTableOptions*/;
 
+-- Forward index, for cache invalidation on file update, etc.
 CREATE UNIQUE INDEX /*i*/il_from ON /*_*/imagelinks (il_from,il_to);
+
+-- Reverse index, for Special:Whatlinkshere and file description page local 
usage
 CREATE INDEX /*i*/il_to ON /*_*/imagelinks (il_to,il_from);
+
+-- Index for Special:Whatlinkshere with namespace filter
 CREATE INDEX /*i*/il_backlinks_namespace ON /*_*/imagelinks 
(il_from_namespace,il_to,il_from);
 
 
@@ -690,9 +737,16 @@
   el_index_60 varbinary(60) NOT N

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Use EtcdConfig

2017-04-10 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/347537 )

Change subject: Use EtcdConfig
..

Use EtcdConfig

Use a MultiConfig which checks etcd first as the main configuration
instance for MW.

Allow $wgReadOnly and $wmfMasterDatacenter to come from etcd. For etcd
changes to $wgReadOnly to take effect, I9bbee946c10742526d342 is needed.

Bug: T156924
Change-Id: I23d4e7da0544e27bb3a4ad84f6f2ef195162fb2c
---
M wmf-config/CommonSettings.php
M wmf-config/InitialiseSettings.php
2 files changed, 16 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/37/347537/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 766c668..fb9b74c 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -51,9 +51,24 @@
 
 set_include_path( "$IP:/usr/local/lib/php:/usr/share/php" );
 
+# Get configuration from etcd
+$wgConfigRegistry = function () {
+   global $wmfDatacenter;
+
+   return new MultiConfig( [
+   new EtcdConfig( [
+   'host' => "_etcd._tcp.$wmfDatacenter.wmnet",
+   'protocol' => 'https'
+   ] ),
+   new GlobalVarConfig
+   ] );
+};
+$wmgConfig = MediaWikiServices::getInstance()->getMainConfig();
+
 # Master datacenter
 # The datacenter from which we serve traffic.
-$wmfMasterDatacenter = 'eqiad';
+$wmfMasterDatacenter = $wmgConfig->has( 'wmfMasterDatacenter' )
+   ? $wmgConfig->get( 'wmfMasterDatacenter' ) : 'eqiad';
 
 ### List of some service hostnames
 # 'meta': meta wiki for user editable content
diff --git a/wmf-config/InitialiseSettings.php 
b/wmf-config/InitialiseSettings.php
index f0e8491..201715e 100644
--- a/wmf-config/InitialiseSettings.php
+++ b/wmf-config/InitialiseSettings.php
@@ -4711,14 +4711,6 @@
'wikivoyage' => 'wikivoyage',
 ],
 
-'wgReadOnly' => [
-   'default' => null,
-],
-
-'wgReadOnlyFile' => [
-   'default' => false,
-],
-
 # wgTranslateNumerals @{
 'wgTranslateNumerals' => [
'default' => true,

-- 
To view, visit https://gerrit.wikimedia.org/r/347537
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I23d4e7da0544e27bb3a4ad84f6f2ef195162fb2c
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: [WIP] A service for read-only mode

2017-04-09 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/347315 )

Change subject: [WIP] A service for read-only mode
..

[WIP] A service for read-only mode

Change-Id: I9bbee946c10742526d3423208efd68cb3cc5a7ee
---
M autoload.php
M includes/GlobalFunctions.php
M includes/MediaWikiServices.php
A includes/ReadOnlyMode.php
M includes/ServiceWiring.php
M includes/WatchedItemStore.php
M includes/db/MWLBFactory.php
M maintenance/rebuildFileCache.php
M maintenance/rebuildImages.php
M tests/phpunit/includes/GlobalFunctions/GlobalTest.php
A tests/phpunit/includes/ReadOnlyModeTest.php
M tests/phpunit/includes/auth/AuthManagerTest.php
12 files changed, 360 insertions(+), 82 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/15/347315/1

diff --git a/autoload.php b/autoload.php
index 3816485..d4b8b2c 100644
--- a/autoload.php
+++ b/autoload.php
@@ -1159,6 +1159,7 @@
'RawAction' => __DIR__ . '/includes/actions/RawAction.php',
'RawMessage' => __DIR__ . '/includes/Message.php',
'ReadOnlyError' => __DIR__ . '/includes/exception/ReadOnlyError.php',
+   'ReadOnlyMode' => __DIR__ . '/includes/ReadOnlyMode.php',
'ReassignEdits' => __DIR__ . '/maintenance/reassignEdits.php',
'RebuildAll' => __DIR__ . '/maintenance/rebuildall.php',
'RebuildFileCache' => __DIR__ . '/maintenance/rebuildFileCache.php',
diff --git a/includes/GlobalFunctions.php b/includes/GlobalFunctions.php
index 3747c23..7eb7193 100644
--- a/includes/GlobalFunctions.php
+++ b/includes/GlobalFunctions.php
@@ -1272,7 +1272,8 @@
  * @return bool
  */
 function wfReadOnly() {
-   return wfReadOnlyReason() !== false;
+   return \MediaWiki\MediaWikiServices::getInstance()->getReadOnlyMode()
+   ->isReadOnly();
 }
 
 /**
@@ -1284,19 +1285,8 @@
  * @return string|bool String when in read-only mode; false otherwise
  */
 function wfReadOnlyReason() {
-   $readOnly = wfConfiguredReadOnlyReason();
-   if ( $readOnly !== false ) {
-   return $readOnly;
-   }
-
-   static $lbReadOnly = null;
-   if ( $lbReadOnly === null ) {
-   // Callers use this method to be aware that data presented to a 
user
-   // may be very stale and thus allowing submissions can be 
problematic.
-   $lbReadOnly = wfGetLB()->getReadOnlyReason();
-   }
-
-   return $lbReadOnly;
+   return \MediaWiki\MediaWikiServices::getInstance()->getReadOnlyMode()
+   ->getReason();
 }
 
 /**
@@ -1306,18 +1296,8 @@
  * @since 1.27
  */
 function wfConfiguredReadOnlyReason() {
-   global $wgReadOnly, $wgReadOnlyFile;
-
-   if ( $wgReadOnly === null ) {
-   // Set $wgReadOnly for faster access next time
-   if ( is_file( $wgReadOnlyFile ) && filesize( $wgReadOnlyFile ) 
> 0 ) {
-   $wgReadOnly = file_get_contents( $wgReadOnlyFile );
-   } else {
-   $wgReadOnly = false;
-   }
-   }
-
-   return $wgReadOnly;
+   return 
\MediaWiki\MediaWikiServices::getInstance()->getConfiguredReadOnlyMode()
+   ->getReason();
 }
 
 /**
diff --git a/includes/MediaWikiServices.php b/includes/MediaWikiServices.php
index e44fefe..5e6bf8f 100644
--- a/includes/MediaWikiServices.php
+++ b/includes/MediaWikiServices.php
@@ -656,6 +656,22 @@
return $this->getService( 'VirtualRESTServiceClient' );
}
 
+   /**
+* @since 1.29
+* @return ReadOnlyMode
+*/
+   public function getConfiguredReadOnlyMode() {
+   return $this->getService( 'ConfiguredReadOnlyMode' );
+   }
+
+   /**
+* @since 1.29
+* @return ReadOnlyMode
+*/
+   public function getReadOnlyMode() {
+   return $this->getService( 'ReadOnlyMode' );
+   }
+

///
// NOTE: When adding a service getter here, don't forget to add a test
// case for it in MediaWikiServicesTest::provideGetters() and in
diff --git a/includes/ReadOnlyMode.php b/includes/ReadOnlyMode.php
new file mode 100644
index 000..820a946
--- /dev/null
+++ b/includes/ReadOnlyMode.php
@@ -0,0 +1,85 @@
+config = $config;
+   }
+
+   public function addLoadBalancer( \Wikimedia\Rdbms\LoadBalancer 
$loadBalancer ) {
+   $this->loadBalancers[] = $loadBalancer;
+   }
+
+   /**
+* Check whether the wiki is in read-only mode.
+*
+* @return bool
+*/
+   public function isReadOnly() {
+   return $this->getReason() !== false;
+   }
+
+   /**
+* Check if the site is in read-only mode and return the message if so
+*
+* This checks the configuration and registered DB load balancers for
+* read-only mode. 

[MediaWiki-commits] [Gerrit] mediawiki/core[REL1_23]: Fix short array syntax in REL1_23 branch

2017-04-06 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/346949 )

Change subject: Fix short array syntax in REL1_23 branch
..

Fix short array syntax in REL1_23 branch

MW 1.23.x only requires PHP 5.3 and so short arrays should not be used.

Change-Id: Ia4071c99d804cafd3cab32ae72b6f196623f28dd
---
M RELEASE-NOTES-1.23
M includes/Title.php
M includes/api/ApiMain.php
3 files changed, 9 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/49/346949/1

diff --git a/RELEASE-NOTES-1.23 b/RELEASE-NOTES-1.23
index 6b77f1f..8ca047a 100644
--- a/RELEASE-NOTES-1.23
+++ b/RELEASE-NOTES-1.23
@@ -1,6 +1,12 @@
 Security reminder: MediaWiki does not require PHP's register_globals. If you
 have it on, turn it '''off''' if you can.
 
+== MediaWiki 1.23.17 ==
+
+=== Changes since 1.23.16 ===
+
+* Fix syntax errors introduced in 1.23.16 when running PHP 5.3.
+
 == MediaWiki 1.23.16 ==
 This is a security and maintenance release of the MediaWiki 1.23 branch.
 
diff --git a/includes/Title.php b/includes/Title.php
index f6625ed..c622723 100644
--- a/includes/Title.php
+++ b/includes/Title.php
@@ -2280,13 +2280,13 @@
} elseif ( $action === 'undelete' ) {
if ( count( $this->getUserPermissionsErrorsInternal( 
'edit', $user, $doExpensiveQueries, true ) ) ) {
// Undeleting implies editing
-   $errors[] = [ 'undelete-cantedit' ];
+   $errors[] = array( 'undelete-cantedit' );
}
if ( !$this->exists()
&& count( 
$this->getUserPermissionsErrorsInternal( 'create', $user, $doExpensiveQueries, 
true ) )
) {
// Undeleting where nothing currently exists 
implies creating
-   $errors[] = [ 'undelete-cantcreate' ];
+   $errors[] = array( 'undelete-cantcreate' );
}
}
return $errors;
diff --git a/includes/api/ApiMain.php b/includes/api/ApiMain.php
index b2c5b89..95f9e9e 100644
--- a/includes/api/ApiMain.php
+++ b/includes/api/ApiMain.php
@@ -143,7 +143,7 @@
private $mCacheMode = 'private';
private $mCacheControl = array();
private $mParamsUsed = array();
-   private $mParamsSensitive = [];
+   private $mParamsSensitive = array();
 
/** @var bool|null Cached return value from 
self::lacksSameOriginSecurity() */
private $lacksSameOriginSecurity = null;

-- 
To view, visit https://gerrit.wikimedia.org/r/346949
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia4071c99d804cafd3cab32ae72b6f196623f28dd
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: REL1_23
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...ParserMigration[wmf/1.29.0-wmf.18]: Don't run tidy twice

2017-04-04 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/346478 )

Change subject: Don't run tidy twice
..

Don't run tidy twice

It defeats the purpose.

Change-Id: Ie6de3842731da2a2d74af03a42de0632085b4c6b
(cherry picked from commit 7b61c8899040393fef10b6a898f086bf3749e6bf)
---
M includes/MigrationEditPage.php
1 file changed, 6 insertions(+), 0 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ParserMigration 
refs/changes/78/346478/1

diff --git a/includes/MigrationEditPage.php b/includes/MigrationEditPage.php
index dc015d7..a42ad14 100644
--- a/includes/MigrationEditPage.php
+++ b/includes/MigrationEditPage.php
@@ -24,6 +24,12 @@
return true;
}
 
+   protected function getPreviewParserOptions() {
+   $parserOptions = parent::getPreviewParserOptions();
+   $parserOptions->setTidy( false );
+   return $parserOptions;
+   }
+
protected function doPreviewParse( \Content $content ) {
$user = $this->context->getUser();
$parserOptions = $this->getPreviewParserOptions();

-- 
To view, visit https://gerrit.wikimedia.org/r/346478
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie6de3842731da2a2d74af03a42de0632085b4c6b
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ParserMigration
Gerrit-Branch: wmf/1.29.0-wmf.18
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...ParserMigration[master]: Don't run tidy twice

2017-04-03 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/346242 )

Change subject: Don't run tidy twice
..

Don't run tidy twice

It defeats the purpose.

Change-Id: Ie6de3842731da2a2d74af03a42de0632085b4c6b
---
M includes/MigrationEditPage.php
1 file changed, 6 insertions(+), 0 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ParserMigration 
refs/changes/42/346242/1

diff --git a/includes/MigrationEditPage.php b/includes/MigrationEditPage.php
index dc015d7..a42ad14 100644
--- a/includes/MigrationEditPage.php
+++ b/includes/MigrationEditPage.php
@@ -24,6 +24,12 @@
return true;
}
 
+   protected function getPreviewParserOptions() {
+   $parserOptions = parent::getPreviewParserOptions();
+   $parserOptions->setTidy( false );
+   return $parserOptions;
+   }
+
protected function doPreviewParse( \Content $content ) {
$user = $this->context->getUser();
$parserOptions = $this->getPreviewParserOptions();

-- 
To view, visit https://gerrit.wikimedia.org/r/346242
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie6de3842731da2a2d74af03a42de0632085b4c6b
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ParserMigration
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix two bugs in installer session error display

2017-04-02 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/346091 )

Change subject: Fix two bugs in installer session error display
..

Fix two bugs in installer session error display

If Installer::startSession() returns false, then showError() is called
before setParserLanguage(), which causes DB access via
Title::getContentModel(). The code which was meant to handle DB access
from the parser did not do so, since the exception class which is thrown
on attempted DB access has changed.

Change-Id: I1478ec06a441851c5336bdde15734642615d45e9
---
M includes/installer/Installer.php
1 file changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/91/346091/1

diff --git a/includes/installer/Installer.php b/includes/installer/Installer.php
index f03fe6a..0a2b808 100644
--- a/includes/installer/Installer.php
+++ b/includes/installer/Installer.php
@@ -446,6 +446,8 @@
$this->parserTitle = Title::newFromText( 'Installer' );
$this->parserOptions = new ParserOptions( $wgUser ); // 
language will be wrong :(
$this->parserOptions->setEditSection( false );
+   // Don't try to access DB before user language is initialised
+   $this->setParserLanguage( Language::factory( 'en' ) );
}
 
/**
@@ -674,7 +676,7 @@
try {
$out = $wgParser->parse( $text, $this->parserTitle, 
$this->parserOptions, $lineStart );
$html = $out->getText();
-   } catch ( DBAccessError $e ) {
+   } catch ( MediaWiki\Services\ServiceDisabledException $e ) {
$html = '  ' . 
htmlspecialchars( $text );
 
if ( !empty( $this->debug ) ) {

-- 
To view, visit https://gerrit.wikimedia.org/r/346091
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1478ec06a441851c5336bdde15734642615d45e9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix incorrect index name in new 1.28.0 installations

2017-04-02 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/346090 )

Change subject: Fix incorrect index name in new 1.28.0 installations
..

Fix incorrect index name in new 1.28.0 installations

This index name should be the same in new installations as in all other
MySQL installations. This is necessary so that
Special:DeletedContributions does not give an SQL error. Release notes
will go in the 1.28 branch.

Change-Id: I535a5513ab56203139c4cf64874ea7e7498990e3
---
M includes/installer/MysqlUpdater.php
A maintenance/archives/patch-rename-ar_usertext_timestamp.sql
2 files changed, 7 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/90/346090/1

diff --git a/includes/installer/MysqlUpdater.php 
b/includes/installer/MysqlUpdater.php
index ff13196..70e790c 100644
--- a/includes/installer/MysqlUpdater.php
+++ b/includes/installer/MysqlUpdater.php
@@ -294,6 +294,7 @@
[ 'addField', 'change_tag', 'ct_id', 
'patch-change_tag-ct_id.sql' ],
[ 'addField', 'tag_summary', 'ts_id', 
'patch-tag_summary-ts_id.sql' ],
[ 'modifyField', 'recentchanges', 'rc_ip', 
'patch-rc_ip_modify.sql' ],
+   [ 'addIndex', 'archive', 'usertext_timestamp', 
'patch-rename-ar_usertext_timestamp.sql' ],
 
// 1.29
[ 'addField', 'externallinks', 'el_index_60', 
'patch-externallinks-el_index_60.sql' ],
diff --git a/maintenance/archives/patch-rename-ar_usertext_timestamp.sql 
b/maintenance/archives/patch-rename-ar_usertext_timestamp.sql
new file mode 100644
index 000..cde96c0
--- /dev/null
+++ b/maintenance/archives/patch-rename-ar_usertext_timestamp.sql
@@ -0,0 +1,6 @@
+-- Rename the archive.ar_usertext_timestamp index to usertext_timestamp.
+-- This is for MySQL only and is only necessary on wikis freshly installed on
+-- 1.28.0 when bug T154872 was present. The patch will probably be removed in
+-- 1.29 since we plan on renaming the index properly to ar_usertext_timestamp.
+ALTER TABLE /*$wgDBprefix*/archive
+   RENAME INDEX ar_usertext_timestamp TO usertext_timestamp;

-- 
To view, visit https://gerrit.wikimedia.org/r/346090
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I535a5513ab56203139c4cf64874ea7e7498990e3
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Update comment for indexName(), explaining why it exists

2017-03-29 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/345494 )

Change subject: Update comment for indexName(), explaining why it exists
..

Update comment for indexName(), explaining why it exists

Bug: T154872
Change-Id: Idf64b85dc56dcc28f3a25b2b8ded44c5490cf56c
---
M includes/libs/rdbms/database/Database.php
1 file changed, 12 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/94/345494/1

diff --git a/includes/libs/rdbms/database/Database.php 
b/includes/libs/rdbms/database/Database.php
index fbfd899..94e2106 100644
--- a/includes/libs/rdbms/database/Database.php
+++ b/includes/libs/rdbms/database/Database.php
@@ -1957,7 +1957,18 @@
 * @return string
 */
protected function indexName( $index ) {
-   // Backwards-compatibility hack
+   // When SQLite indexes were introduced in r45764, it was noted 
that
+   // SQLite requires index names to be unique within the whole 
database,
+   // not just within a schema. As discussed in CR r45819, to 
avoid the
+   // need for a schema change on existing installations, the 
indexes
+   // were implicitly mapped from the new names to the old names.
+   //
+   // This mapping can be removed if DB patches are introduced to 
alter
+   // the relevant tables in existing installations. Note that 
because
+   // this index mapping applies to table creation, even new 
installations
+   // of MySQL have the old names (except for installations 
created during
+   // a period where this mapping was inappropriately removed, see
+   // T154872).
$renamed = [
'ar_usertext_timestamp' => 'usertext_timestamp',
'un_user_id' => 'user_id',

-- 
To view, visit https://gerrit.wikimedia.org/r/345494
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Idf64b85dc56dcc28f3a25b2b8ded44c5490cf56c
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: RemexCompatMunger: fix a couple of memory leaks

2017-03-22 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/344289 )

Change subject: RemexCompatMunger: fix a couple of memory leaks
..

RemexCompatMunger: fix a couple of memory leaks

Change-Id: I47578b3f73320e84a157417c288de97b5d26e18f
---
M includes/tidy/RemexCompatMunger.php
1 file changed, 8 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/89/344289/1

diff --git a/includes/tidy/RemexCompatMunger.php 
b/includes/tidy/RemexCompatMunger.php
index d5f5c28..dbcf568 100644
--- a/includes/tidy/RemexCompatMunger.php
+++ b/includes/tidy/RemexCompatMunger.php
@@ -229,7 +229,6 @@
 * @param integer $sourceStart
 * @param integer $sourceLength
 */
-
public function insertElement( $preposition, $refElement, Element 
$element, $void,
$sourceStart, $sourceLength
) {
@@ -248,9 +247,10 @@
$newParent = $this->serializer->getParentNode( $parent 
);
$parent = $newParent;
$parentData = $parent->snData;
+   $pElement = $parentData->childPElement;
$parentData->childPElement = null;
$newRef = $refElement->userData;
-   // FIXME cannot call endTag() since we don't have an 
Element
+   $this->endTag( $pElement, $sourceStart, 0 );
} elseif ( $under && $parentData->isSplittable
&& (bool)$parentData->ancestorPNode !== $inline
) {
@@ -425,7 +425,13 @@
}
 
public function endTag( Element $element, $sourceStart, $sourceLength ) 
{
+   $data = $element->userData->snData;
+   if ( $data->childPElement ) {
+   $this->endTag( $data->childPElement, $sourceStart, 0 );
+   }
$this->serializer->endTag( $element, $sourceStart, 
$sourceLength );
+   $element->userData->snData = null;
+   $element->userData = null;
}
 
public function doctype( $name, $public, $system, $quirks, 
$sourceStart, $sourceLength ) {

-- 
To view, visit https://gerrit.wikimedia.org/r/344289
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I47578b3f73320e84a157417c288de97b5d26e18f
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Add benchmarkTidy.php, to benchmark tidy drivers

2017-03-22 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/344290 )

Change subject: Add benchmarkTidy.php, to benchmark tidy drivers
..

Add benchmarkTidy.php, to benchmark tidy drivers

Change-Id: I254793fc55c57a98c07ae1e4c27e6005965c9a20
---
M includes/parser/MWTidy.php
A maintenance/benchmarks/benchmarkTidy.php
2 files changed, 76 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/90/344290/1

diff --git a/includes/parser/MWTidy.php b/includes/parser/MWTidy.php
index 01bf2d0..ffc884e 100644
--- a/includes/parser/MWTidy.php
+++ b/includes/parser/MWTidy.php
@@ -83,7 +83,7 @@
/**
 * @return bool|\MediaWiki\Tidy\TidyDriverBase
 */
-   protected static function singleton() {
+   public static function singleton() {
global $wgUseTidy, $wgTidyInternal, $wgTidyConf, $wgDebugTidy, 
$wgTidyConfig,
$wgTidyBin, $wgTidyOpts;
 
diff --git a/maintenance/benchmarks/benchmarkTidy.php 
b/maintenance/benchmarks/benchmarkTidy.php
new file mode 100644
index 000..3219e6c
--- /dev/null
+++ b/maintenance/benchmarks/benchmarkTidy.php
@@ -0,0 +1,75 @@
+addOption( 'file', 'A filename which contains the input 
text', true, true );
+   $this->addOption( 'driver', 'The Tidy driver name, or false to 
use the configured instance',
+   false,  true );
+   $this->addOption( 'tidy-config', 'JSON encoded value for the 
tidy configuration array',
+   false, true );
+   }
+
+   public function execute() {
+   $html = file_get_contents( $this->getOption( 'file' ) );
+   if ( $html === false ) {
+   $this->error( "Unable to open input file", 1 );
+   }
+   if ( $this->hasOption( 'driver' ) || $this->hasOption( 
'tidy-config' ) ) {
+   $config = json_decode( $this->getOption( 'tidy-config', 
'{}' ), true );
+   if ( !is_array( $config ) ) {
+   $this->error( "Invalid JSON tidy config", 1 );
+   }
+   $config += [ 'driver' => $this->getOption( 'driver', 
'RemexHtml' ) ];
+   $driver = MWTidy::factory( $config );
+   } else {
+   $driver = MWTidy::singleton();
+   }
+
+   $this->benchmark( $driver, $html );
+   }
+
+   private function benchmark( $driver, $html ) {
+   global $wgContLang;
+
+   $times = [];
+   $innerCount = 10;
+   $outerCount = 10;
+   for ( $j = 1; $j <= $outerCount; $j++ ) {
+   $t = microtime( true );
+   for ( $i = 0; $i < $innerCount; $i++ ) {
+   $driver->tidy( $html );
+   print $wgContLang->formatSize( 
memory_get_usage( true ) ) . "\n";
+   }
+   $t = ( ( microtime( true ) - $t ) / $innerCount ) * 
1000;
+   $times[] = $t;
+   print "Run $j: $t\n";
+   }
+   print "\n";
+
+   sort( $times, SORT_NUMERIC );
+   $n = $outerCount;
+   $min = $times[0];
+   $max = end( $times );
+   if ( $n % 2 ) {
+   $median = $times[ ( $n - 1 ) / 2 ];
+   } else {
+   $median = ( $times[$n / 2] + $times[$n / 2 - 1] ) / 2;
+   }
+   $mean = array_sum( $times ) / $n;
+
+   print "Minimum: $min ms\n";
+   print "Median: $median ms\n";
+   print "Mean: $mean ms\n";
+   print "Maximum: $max ms\n";
+   print "Memory usage: " .
+   $wgContLang->formatSize( memory_get_usage( true ) ) . 
"\n";
+   print "Peak memory usage: " .
+   $wgContLang->formatSize( memory_get_peak_usage( true ) 
) . "\n";
+   }
+}
+
+$maintClass = 'BenchmarkTidy';
+require RUN_MAINTENANCE_IF_MAIN;

-- 
To view, visit https://gerrit.wikimedia.org/r/344290
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I254793fc55c57a98c07ae1e4c27e6005965c9a20
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Update RemexHtml to 1.0.1

2017-03-22 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/344288 )

Change subject: Update RemexHtml to 1.0.1
..

Update RemexHtml to 1.0.1

For a bug fix

Change-Id: I5285c0b1c6f42891c002637cd3e4bd2f54615940
---
M composer.json
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/88/344288/1

diff --git a/composer.json b/composer.json
index df699bc..cd8f310 100644
--- a/composer.json
+++ b/composer.json
@@ -38,7 +38,7 @@
"wikimedia/ip-set": "1.1.0",
"wikimedia/php-session-serializer": "1.0.4",
"wikimedia/relpath": "1.0.3",
-   "wikimedia/remex-html": "1.0.0",
+   "wikimedia/remex-html": "1.0.1",
"wikimedia/running-stat": "1.1.0",
"wikimedia/scoped-callback": "1.0.0",
"wikimedia/utfnormal": "1.1.0",

-- 
To view, visit https://gerrit.wikimedia.org/r/344288
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5285c0b1c6f42891c002637cd3e4bd2f54615940
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...release[master]: Add ParserMigration

2017-03-22 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/344280 )

Change subject: Add ParserMigration
..

Add ParserMigration

Change-Id: I0afc58e50f77e67e851bc860ee8198a5607a38e4
---
M make-wmf-branch/config.json
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/tools/release 
refs/changes/80/344280/1

diff --git a/make-wmf-branch/config.json b/make-wmf-branch/config.json
index 6a770ee..d136626 100644
--- a/make-wmf-branch/config.json
+++ b/make-wmf-branch/config.json
@@ -100,6 +100,7 @@
"PageTriage",
"PageViewInfo",
"ParserFunctions",
+   "ParserMigration",
"ParsoidBatchAPI",
"PdfHandler",
"Petition",

-- 
To view, visit https://gerrit.wikimedia.org/r/344280
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0afc58e50f77e67e851bc860ee8198a5607a38e4
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/tools/release
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Deploy ParserMigration extension

2017-03-22 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/344276 )

Change subject: Deploy ParserMigration extension
..

Deploy ParserMigration extension

A tool to compare RemexHtml output with Tidy. RemexHtml is already
available in vendor/ in 1.29.0-wmf.14 and later.

Also fix a comment which was accidentally moved away from its correct
location.

Bug: T141586
Change-Id: Id3c43149d8fecd0cfaa028403c64e7005db40144
---
M wmf-config/CommonSettings.php
M wmf-config/InitialiseSettings.php
M wmf-config/extension-list
3 files changed, 21 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/76/344276/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 5cc2092..cf4be9a 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -3515,12 +3515,25 @@
};
 }
 
-# THIS MUST BE AFTER ALL EXTENSIONS ARE INCLUDED
-#
-# REALLY ... we're not kidding here ... NO EXTENSIONS AFTER
+if ( $wmgUseParserMigration ) {
+   wfLoadExtension( 'ParserMigration' );
+   $wgParserMigrationTidiers = [
+   [
+   'driver' => 'RaggettInternalHHVM',
+   'tidyConfigFile' => $wgTidyConf,
+   ],
+   [
+   'driver' => 'RemexHtml',
+   ],
+   ];
+}
 
 if ( $wmfRealm === 'labs' ) {
require( "$wmfConfigDir/CommonSettings-labs.php" );
 }
 
+# THIS MUST BE AFTER ALL EXTENSIONS ARE INCLUDED
+#
+# REALLY ... we're not kidding here ... NO EXTENSIONS AFTER
+
 require( "$wmfConfigDir/ExtensionMessages-$wmgVersionNumber.php" );
diff --git a/wmf-config/InitialiseSettings.php 
b/wmf-config/InitialiseSettings.php
index 116d96a..98db17e 100644
--- a/wmf-config/InitialiseSettings.php
+++ b/wmf-config/InitialiseSettings.php
@@ -18385,6 +18385,10 @@
'foundationwiki' => true,
 ],
 
+'wmgUseParserMigration' => [
+   'default' => true,
+],
+
 ];
 
 ### WMF Labs override #
diff --git a/wmf-config/extension-list b/wmf-config/extension-list
index 74c0728..b6b725f 100644
--- a/wmf-config/extension-list
+++ b/wmf-config/extension-list
@@ -93,6 +93,7 @@
 $IP/extensions/PageTriage/PageTriage.php
 $IP/extensions/PageViewInfo/extension.json
 $IP/extensions/ParserFunctions/extension.json
+$IP/extensions/ParserMigration/extension.json
 $IP/extensions/ParsoidBatchAPI/extension.json
 $IP/extensions/PdfHandler/extension.json
 $IP/extensions/Petition/extension.json

-- 
To view, visit https://gerrit.wikimedia.org/r/344276
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id3c43149d8fecd0cfaa028403c64e7005db40144
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...Scribunto[master]: Make the maximum language cache size configurable

2017-03-19 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/343590 )

Change subject: Make the maximum language cache size configurable
..

Make the maximum language cache size configurable

Make the language cache size configurable, and increase the default from
20 to 30. It needs to be fairly small on default installations, but can
be essentially unlimited if $wgLocalisationCacheConf['manualRecache'] is
true.

Bug: T85461
Change-Id: Idb17691b30b0d2565a1624e5159df7d9b795764d
---
M Scribunto.php
M common/Base.php
M engines/LuaCommon/LanguageLibrary.php
3 files changed, 21 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Scribunto 
refs/changes/90/343590/1

diff --git a/Scribunto.php b/Scribunto.php
index 15a4c59..39b15bc 100644
--- a/Scribunto.php
+++ b/Scribunto.php
@@ -168,6 +168,13 @@
// implemented in the future, and there is no guarantee that a
// simulation of setfenv() and getfenv() will be provided.
'allowEnvFuncs' => false,
+
+   // The maximum number of languages about which data can be 
requested.
+   // The cost is about 1.5MB of memory usage per language on 
default
+   // installations (during recache), but if recaching is disabled 
with
+   // $wgLocalisationCacheConf['manualRecache'] = false
+   // then memory usage is perhaps 10x smaller.
+   'maxLangCacheSize' => 30,
),
'luastandalone' => array(
'class' => 'Scribunto_LuaStandaloneEngine',
@@ -182,6 +189,7 @@
'memoryLimit' => 50 * 1024 * 1024,
'cpuLimit' => 7,
'allowEnvFuncs' => false,
+   'maxLangCacheSize' => 30,
),
 );
 
diff --git a/common/Base.php b/common/Base.php
index 180c375..33c8a9d 100644
--- a/common/Base.php
+++ b/common/Base.php
@@ -123,6 +123,16 @@
}
 
/**
+* Get an element from the configuration array
+*
+* @param string $optionName
+* @return mixed
+*/
+   public function getOption( $optionName ) {
+   return isset( $optionName ) ? $this->options[$optionName] : 
null;
+   }
+
+   /**
 * @param string $message
 * @param $params array
 * @return ScribuntoException
diff --git a/engines/LuaCommon/LanguageLibrary.php 
b/engines/LuaCommon/LanguageLibrary.php
index dbf205e..e33934e 100644
--- a/engines/LuaCommon/LanguageLibrary.php
+++ b/engines/LuaCommon/LanguageLibrary.php
@@ -6,11 +6,13 @@
 
public $langCache = array();
public $timeCache = array();
+   public $maxLangCacheSize;
 
function register() {
// Pre-populate the language cache
global $wgContLang;
$this->langCache[$wgContLang->getCode()] = $wgContLang;
+   $this->maxLangCacheSize = $this->getEngine()->getOption( 
'maxLangCacheSize' );
 
$statics = array(
'getContLangCode',
@@ -111,7 +113,7 @@
$name = strval( $name );
$code = array_shift( $args );
if ( !isset( $this->langCache[$code] ) ) {
-   if ( count( $this->langCache ) > 
self::MAX_LANG_CACHE_SIZE ) {
+   if ( count( $this->langCache ) > 
$this->maxLangCacheSize ) {
throw new Scribunto_LuaError( 'too many 
language codes requested' );
}
try {

-- 
To view, visit https://gerrit.wikimedia.org/r/343590
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Idb17691b30b0d2565a1624e5159df7d9b795764d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Scribunto
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Basic PHPLint support

2017-03-13 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/342562 )

Change subject: Basic PHPLint support
..

Basic PHPLint support

Move src to a directory named after the namespace RemexHtml, to support
PHPLint's idea of autoloading.

Ran PHPLint 3.0 and fixed a few minor issues. There are still many, many
errors, and it's not a goal to fix them. But I was able to verify that there
are no unintentional missing breaks, or variables accessed before their
assignment.

Change-Id: Icb269d934a80bdfe0332c0003c854bffdc58eb71
---
R RemexHtml/DOM/DOMBuilder.php
R RemexHtml/DOM/DOMFormatter.php
R RemexHtml/DOM/DOMSerializer.php
R RemexHtml/DOM/DOMUtils.php
R RemexHtml/FuzzTest/CharacterSalad.php
R RemexHtml/FuzzTest/FuzzData.php
R RemexHtml/FuzzTest/FuzzTest.php
R RemexHtml/FuzzTest/Html5Depurate.php
R RemexHtml/FuzzTest/TokenSalad.php
R RemexHtml/FuzzTest/Utils.php
R RemexHtml/FuzzTest/validatornu-dictionary
R RemexHtml/GenerateDataFiles.php
R RemexHtml/HTMLData.php
R RemexHtml/PropGuard.php
R RemexHtml/Serializer/AbstractSerializer.php
R RemexHtml/Serializer/DepurateFormatter.php
R RemexHtml/Serializer/FastFormatter.php
R RemexHtml/Serializer/Formatter.php
R RemexHtml/Serializer/HtmlFormatter.php
R RemexHtml/Serializer/Serializer.php
R RemexHtml/Serializer/SerializerError.php
R RemexHtml/Serializer/SerializerNode.php
R RemexHtml/Serializer/TestFormatter.php
R RemexHtml/Tokenizer/Attribute.php
R RemexHtml/Tokenizer/Attributes.php
R RemexHtml/Tokenizer/LazyAttributes.php
R RemexHtml/Tokenizer/PlainAttributes.php
R RemexHtml/Tokenizer/TestTokenHandler.php
R RemexHtml/Tokenizer/TokenGenerator.php
R RemexHtml/Tokenizer/TokenGeneratorHandler.php
R RemexHtml/Tokenizer/TokenHandler.php
R RemexHtml/Tokenizer/TokenSerializer.php
R RemexHtml/Tokenizer/Tokenizer.php
R RemexHtml/Tokenizer/TokenizerError.php
R RemexHtml/TreeBuilder/ActiveFormattingElements.php
R RemexHtml/TreeBuilder/AfterAfterBody.php
R RemexHtml/TreeBuilder/AfterAfterFrameset.php
R RemexHtml/TreeBuilder/AfterBody.php
R RemexHtml/TreeBuilder/AfterFrameset.php
R RemexHtml/TreeBuilder/AfterHead.php
R RemexHtml/TreeBuilder/BeforeHead.php
R RemexHtml/TreeBuilder/BeforeHtml.php
R RemexHtml/TreeBuilder/CachingStack.php
R RemexHtml/TreeBuilder/DestructTracer.php
R RemexHtml/TreeBuilder/DestructTracerNode.php
R RemexHtml/TreeBuilder/DispatchTracer.php
R RemexHtml/TreeBuilder/Dispatcher.php
R RemexHtml/TreeBuilder/Element.php
R RemexHtml/TreeBuilder/ForeignAttributes.php
R RemexHtml/TreeBuilder/FormattingElement.php
R RemexHtml/TreeBuilder/InBody.php
R RemexHtml/TreeBuilder/InCaption.php
R RemexHtml/TreeBuilder/InCell.php
R RemexHtml/TreeBuilder/InColumnGroup.php
R RemexHtml/TreeBuilder/InForeignContent.php
R RemexHtml/TreeBuilder/InFrameset.php
R RemexHtml/TreeBuilder/InHead.php
R RemexHtml/TreeBuilder/InHeadNoscript.php
R RemexHtml/TreeBuilder/InPre.php
R RemexHtml/TreeBuilder/InRow.php
R RemexHtml/TreeBuilder/InSelect.php
R RemexHtml/TreeBuilder/InSelectInTable.php
R RemexHtml/TreeBuilder/InTable.php
R RemexHtml/TreeBuilder/InTableBody.php
R RemexHtml/TreeBuilder/InTableText.php
R RemexHtml/TreeBuilder/InTemplate.php
R RemexHtml/TreeBuilder/InTextarea.php
R RemexHtml/TreeBuilder/Initial.php
R RemexHtml/TreeBuilder/InsertionMode.php
R RemexHtml/TreeBuilder/Marker.php
R RemexHtml/TreeBuilder/SimpleStack.php
R RemexHtml/TreeBuilder/Stack.php
R RemexHtml/TreeBuilder/TemplateModeStack.php
R RemexHtml/TreeBuilder/Text.php
R RemexHtml/TreeBuilder/TreeBuilder.php
R RemexHtml/TreeBuilder/TreeBuilderError.php
R RemexHtml/TreeBuilder/TreeHandler.php
R RemexHtml/TreeBuilder/TreeMutationTracer.php
M composer.json
A tests/phplint/autoload.php
A tests/phplint/phplint
81 files changed, 45 insertions(+), 16 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/62/342562/1

diff --git a/src/DOM/DOMBuilder.php b/RemexHtml/DOM/DOMBuilder.php
similarity index 100%
rename from src/DOM/DOMBuilder.php
rename to RemexHtml/DOM/DOMBuilder.php
diff --git a/src/DOM/DOMFormatter.php b/RemexHtml/DOM/DOMFormatter.php
similarity index 87%
rename from src/DOM/DOMFormatter.php
rename to RemexHtml/DOM/DOMFormatter.php
index dad8a7d..2a84f0e 100644
--- a/src/DOM/DOMFormatter.php
+++ b/RemexHtml/DOM/DOMFormatter.php
@@ -14,7 +14,7 @@
/**
 * Non-recursively format a DOMElement.
 *
-* @param DOMElement $Element The element to format
+* @param DOMElement $element The element to format
 * @param string $contents The formatted contents of the element
 */
function formatDOMElement( \DOMElement $element, $contents );
diff --git a/src/DOM/DOMSerializer.php b/RemexHtml/DOM/DOMSerializer.php
similarity index 100%
rename from src/DOM/DOMSerializer.php
rename to RemexHtml/DOM/DOMSerializer.php
diff --git a/src/DOM/DOMUtils.php b/RemexHtml/DOM/DOMUtils.php
similarity index 100%
rename from src/DOM/DOMUtils.php
renam

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Added testing for error counts, and disabled it

2017-03-13 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/342556 )

Change subject: Added testing for error counts, and disabled it
..

Added testing for error counts, and disabled it

Added tests for error counts, fixed a couple of resulting issues.
Reconsidered the utility of this approach, and hid the feature behind a
hard-coded option.

Change-Id: Ia9d36da83f9d3bf4c78a9efcf160f171e38fa3b3
---
M src/Tokenizer/Tokenizer.php
M src/TreeBuilder/InTable.php
M src/TreeBuilder/TreeBuilder.php
M tests/phpunit/TokenizerTest.php
M tests/phpunit/TreeBuilderTest.php
5 files changed, 48 insertions(+), 14 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/56/342556/1

diff --git a/src/Tokenizer/Tokenizer.php b/src/Tokenizer/Tokenizer.php
index 78b2383..0d84b15 100644
--- a/src/Tokenizer/Tokenizer.php
+++ b/src/Tokenizer/Tokenizer.php
@@ -675,9 +675,6 @@
 
if ( isset( $m[self::MD_DT_PUBLIC_WS] ) && 
$m[self::MD_DT_PUBLIC_WS][1] >= 0 ) {
// PUBLIC keyword found
-   if ( !$igerr && !$eof && !strlen( 
$m[self::MD_DT_PUBLIC_WS][0] ) ) {
-   $this->error( 'missing whitespace', 
$m[self::MD_DT_PUBLIC_WS][1] );
-   }
$public = $this->interpretDoctypeQuoted( $m,
self::MD_DT_PUBLIC_DQ, self::MD_DT_PUBLIC_SQ, 
$quirks );
if ( $public === null ) {
@@ -685,6 +682,8 @@
if ( !$eof && !$igerr ) {
$this->error( 'missing public 
identifier', $m[self::MD_DT_PUBLIC_WS][1] );
}
+   } elseif ( !$igerr && !$eof && !strlen( 
$m[self::MD_DT_PUBLIC_WS][0] ) ) {
+   $this->error( 'missing whitespace', 
$m[self::MD_DT_PUBLIC_WS][1] );
}
 
// Check for a system ID after the public ID
@@ -699,14 +698,13 @@
}
} elseif ( isset( $m[self::MD_DT_SYSTEM_WS] ) && 
$m[self::MD_DT_SYSTEM_WS][1] >= 0 ) {
// SYSTEM keyword found
-   if ( !$igerr && !strlen( $m[self::MD_DT_SYSTEM_WS][0] ) 
) {
-   $this->error( 'missing whitespace', 
$m[self::MD_DT_SYSTEM_WS][1] );
-   }
$system = $this->interpretDoctypeQuoted( $m,
self::MD_DT_SYSTEM_DQ, self::MD_DT_SYSTEM_SQ, 
$quirks );
if ( $system === null ) {
$quirks = true;
$this->error( 'missing system identifier', 
$m[self::MD_DT_SYSTEM_WS][1] );
+   } elseif ( !$igerr && !strlen( 
$m[self::MD_DT_SYSTEM_WS][0] ) ) {
+   $this->error( 'missing whitespace', 
$m[self::MD_DT_SYSTEM_WS][1] );
}
 
}
diff --git a/src/TreeBuilder/InTable.php b/src/TreeBuilder/InTable.php
index 4b03fb0..5827c7f 100644
--- a/src/TreeBuilder/InTable.php
+++ b/src/TreeBuilder/InTable.php
@@ -168,6 +168,7 @@
break;
 
default:
+   $builder->error( 'unexpected end tag in table, 
fostering', $sourceStart );
$builder->fosterParenting = true;
$dispatcher->inBody->endTag( $name, $sourceStart, 
$sourceLength );
$builder->fosterParenting = false;
diff --git a/src/TreeBuilder/TreeBuilder.php b/src/TreeBuilder/TreeBuilder.php
index 3523773..fe2a212 100644
--- a/src/TreeBuilder/TreeBuilder.php
+++ b/src/TreeBuilder/TreeBuilder.php
@@ -252,13 +252,22 @@
 * @param array $allowed An array with the HTML element names in the key
 */
public function checkUnclosed( $allowed, $pos ) {
+   if ( $this->ignoreErrors ) {
+   return;
+   }
+
$stack = $this->stack;
+   $unclosedErrors = [];
for ( $i = $stack->length() - 1; $i >= 0; $i-- ) {
$unclosedName = $stack->item( $i )->htmlName;
if ( !isset( $allowed[$unclosedName] ) ) {
-   $this->error( "closing unclosed 
<$unclosedName>", $pos );
+   $unclosedErrors[$unclosedName] = true;
}
}
+   if ( $unclosedErrors ) {
+   $names = implode( ', ', array_keys( $unclosedErrors ) );
+   $this->error( "closing unclosed $names", $pos );
+   }
}
 
/**
diff --git a/tests/phpunit/TokenizerTest.php b/tests/phpunit/TokenizerTest.php
index 5322471..db9cdb9 100644
--- a/tests/phpunit/TokenizerTest.php
+

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Fix missing breaks in switch statements

2017-03-12 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/342413 )

Change subject: Fix missing breaks in switch statements
..

Fix missing breaks in switch statements

What clown wrote this code, eh?

Testing is difficult without asserting error counts, which might be
added in a subsequent commit. Hopefully I now found all the missing
breaks by eye, although I haven't yet found a tool to automatically
detect them.

Change-Id: I118e51fec864db1f057b3609c2446b7622a35bbf
---
M src/TreeBuilder/InCell.php
M src/TreeBuilder/InRow.php
M src/TreeBuilder/InTableBody.php
M tests/local/tree-construction/remex01.dat
4 files changed, 45 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/13/342413/1

diff --git a/src/TreeBuilder/InCell.php b/src/TreeBuilder/InCell.php
index 73798a0..f7f4003 100644
--- a/src/TreeBuilder/InCell.php
+++ b/src/TreeBuilder/InCell.php
@@ -32,6 +32,7 @@
}
$this->closeTheCell( $sourceStart )
->startTag( $name, $attrs, $selfClose, 
$sourceStart, $sourceLength );
+   break;
 
default:
$this->dispatcher->inBody->startTag(
diff --git a/src/TreeBuilder/InRow.php b/src/TreeBuilder/InRow.php
index 477c204..df4c8d6 100644
--- a/src/TreeBuilder/InRow.php
+++ b/src/TreeBuilder/InRow.php
@@ -85,6 +85,8 @@
$builder->pop( $sourceStart, $sourceLength );
$dispatcher->switchMode( Dispatcher::IN_TABLE_BODY )
->endTag( $name, $sourceStart, $sourceLength );
+   break;
+
case 'tbody':
case 'tfoot':
case 'thead':
@@ -100,6 +102,7 @@
$builder->pop( $sourceStart, 0 );
$dispatcher->switchMode( Dispatcher::IN_TABLE_BODY )
->endTag( $name, $sourceStart, $sourceLength );
+   break;
 
case 'body':
case 'caption':
diff --git a/src/TreeBuilder/InTableBody.php b/src/TreeBuilder/InTableBody.php
index 145a5d3..0547f45 100644
--- a/src/TreeBuilder/InTableBody.php
+++ b/src/TreeBuilder/InTableBody.php
@@ -84,6 +84,7 @@
$builder->clearStackBack( self::$tableBodyContext, 
$sourceStart );
$builder->pop( $sourceStart, $sourceLength );
$dispatcher->switchMode( Dispatcher::IN_TABLE );
+   break;
 
case 'body':
case 'caption':
diff --git a/tests/local/tree-construction/remex01.dat 
b/tests/local/tree-construction/remex01.dat
index b884384..4dabfe2 100644
--- a/tests/local/tree-construction/remex01.dat
+++ b/tests/local/tree-construction/remex01.dat
@@ -11,3 +11,43 @@
 | 
 |   "
 x"
+
+#data
+
+#errors
+[0] missing doctype
+#document
+| 
+|   
+|   
+| 
+|   
+| 
+|   
+| 
+|   
+| 
+|   id="1"
+| 
+|   id="2"
+
+#data
+
+#errors
+#document
+| 
+| 
+|   
+|   
+| 
+|   
+| 
+|   
+| 
+|   
+| 
+|   id="1"
+|   
+| id="2"
+| 
+|   id="3"

-- 
To view, visit https://gerrit.wikimedia.org/r/342413
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I118e51fec864db1f057b3609c2446b7622a35bbf
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/libs/RemexHtml
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Fix utfnormal dependency to 1.1.0

2017-03-12 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/342412 )

Change subject: Fix utfnormal dependency to 1.1.0
..

Fix utfnormal dependency to 1.1.0

Per MW policy

Change-Id: I2567974fcbcfc1ec71ac24b604324b8c98638712
---
M composer.json
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/12/342412/1

diff --git a/composer.json b/composer.json
index 356c9dd..48f0d45 100644
--- a/composer.json
+++ b/composer.json
@@ -10,7 +10,7 @@
],
"require": {
"ext-mbstring": "*",
-   "wikimedia/utfnormal": "1.*"
+   "wikimedia/utfnormal": "1.1.0"
},
"require-dev": {
"phpunit/phpunit": "4.*",

-- 
To view, visit https://gerrit.wikimedia.org/r/342412
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2567974fcbcfc1ec71ac24b604324b8c98638712
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/libs/RemexHtml
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: RemexDriver: optional decoding of character references

2017-03-09 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/342151 )

Change subject: RemexDriver: optional decoding of character references
..

RemexDriver: optional decoding of character references

Decoding character references takes time, but may avoid some whitespace
changes in visual diff testing. The terminology here matches RemexHtml.

Change-Id: I1df165c6be277098b0993f333432e0e28815e49f
---
M includes/tidy/RemexCompatFormatter.php
M includes/tidy/RemexDriver.php
2 files changed, 16 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/51/342151/1

diff --git a/includes/tidy/RemexCompatFormatter.php 
b/includes/tidy/RemexCompatFormatter.php
index 3dc727b..dd8c610 100644
--- a/includes/tidy/RemexCompatFormatter.php
+++ b/includes/tidy/RemexCompatFormatter.php
@@ -19,10 +19,16 @@
 
public function __construct( $options = [] ) {
parent::__construct( $options );
+   $options += [
+   'ignoreCharRefs' => true
+   ];
+
$this->attributeEscapes["\xc2\xa0"] = ' ';
-   unset( $this->attributeEscapes["&"] );
$this->textEscapes["\xc2\xa0"] = ' ';
-   unset( $this->textEscapes["&"] );
+   if ( $options['ignoreCharRefs'] ) {
+   unset( $this->attributeEscapes["&"] );
+   unset( $this->textEscapes["&"] );
+   }
}
 
public function startDocument( $fragmentNamespace, $fragmentName ) {
diff --git a/includes/tidy/RemexDriver.php b/includes/tidy/RemexDriver.php
index e02af88..57aa5cb 100644
--- a/includes/tidy/RemexDriver.php
+++ b/includes/tidy/RemexDriver.php
@@ -11,19 +11,24 @@
 class RemexDriver extends TidyDriverBase {
private $trace;
private $pwrap;
+   private $ignoreCharRefs;
 
public function __construct( array $config ) {
$config += [
'treeMutationTrace' => false,
-   'pwrap' => true
+   'pwrap' => true,
+   'ignoreCharRefs' => true,
];
$this->trace = $config['treeMutationTrace'];
$this->pwrap = $config['pwrap'];
+   $this->ignoreCharRefs = $config['ignoreCharRefs'];
parent::__construct( $config );
}
 
public function tidy( $text ) {
-   $formatter = new RemexCompatFormatter;
+   $formatter = new RemexCompatFormatter( [
+   'ignoreCharRefs' => $this->ignoreCharRefs
+   ] );
$serializer = new Serializer( $formatter );
if ( $this->pwrap ) {
$munger = new RemexCompatMunger( $serializer );
@@ -44,7 +49,7 @@
$dispatcher = new Dispatcher( $treeBuilder );
$tokenizer = new Tokenizer( $dispatcher, $text, [
'ignoreErrors' => true,
-   'ignoreCharRefs' => true,
+   'ignoreCharRefs' => $this->ignoreCharRefs,
'ignoreNulls' => true,
'skipPreprocess' => true,
] );

-- 
To view, visit https://gerrit.wikimedia.org/r/342151
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1df165c6be277098b0993f333432e0e28815e49f
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Add RemexHtml to the list of available Tidy drivers

2017-03-08 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/341921 )

Change subject: Add RemexHtml to the list of available Tidy drivers
..

Add RemexHtml to the list of available Tidy drivers

Change-Id: I5a87a6ed24ca3ef7c5fdb21e74f9eb410bf74b4c
---
M includes/DefaultSettings.php
M includes/parser/MWTidy.php
2 files changed, 5 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/21/341921/1

diff --git a/includes/DefaultSettings.php b/includes/DefaultSettings.php
index c483366..7fe7c60 100644
--- a/includes/DefaultSettings.php
+++ b/includes/DefaultSettings.php
@@ -4235,7 +4235,8 @@
  *- RaggettInternalPHP: Use the PECL extension
  *- RaggettExternal: Shell out to an external binary (tidyBin)
  *- Html5Depurate: Use external Depurate service
- *- Html5Internal: Use the built-in HTML5 balancer
+ *- Html5Internal: Use the Balancer library in PHP
+ *- RemexHtml: Use the RemexHtml library in PHP
  *
  *  - tidyConfigFile: Path to configuration file for any of the Raggett drivers
  *  - debugComment: True to add a comment to the output with warning messages
diff --git a/includes/parser/MWTidy.php b/includes/parser/MWTidy.php
index 5e54615..01bf2d0 100644
--- a/includes/parser/MWTidy.php
+++ b/includes/parser/MWTidy.php
@@ -138,6 +138,9 @@
case 'Html5Internal':
$instance = new MediaWiki\Tidy\Html5Internal( 
$config );
break;
+   case 'RemexHtml':
+   $instance = new MediaWiki\Tidy\RemexDriver( 
$config );
+   break;
case 'disabled':
return false;
default:

-- 
To view, visit https://gerrit.wikimedia.org/r/341921
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5a87a6ed24ca3ef7c5fdb21e74f9eb410bf74b4c
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix @covers for FileBackend

2017-02-26 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340065 )

Change subject: Fix @covers for FileBackend
..

Fix @covers for FileBackend

It's up to the developer to choose the level of granularity they are
aspiring for in their test coverage. But that granularity level should be
reflected in @covers. When you have hundreds of lines of code testing a
single-line function (FileBackend::doOperation), that's a good hint that
something went wrong.

FileBackendTest is 2700 lines of code and appears to be aiming to test
the whole filebackend module. So I adjusted @covers to reflect that.

Change-Id: Iacd8cf475d1761c3b32c739983343619a9509d6b
---
M tests/phpunit/includes/filebackend/FileBackendTest.php
M tests/phpunit/includes/filebackend/SwiftFileBackendTest.php
2 files changed, 43 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/65/340065/1

diff --git a/tests/phpunit/includes/filebackend/FileBackendTest.php 
b/tests/phpunit/includes/filebackend/FileBackendTest.php
index c3d31d1..0a55760 100644
--- a/tests/phpunit/includes/filebackend/FileBackendTest.php
+++ b/tests/phpunit/includes/filebackend/FileBackendTest.php
@@ -4,6 +4,44 @@
  * @group FileRepo
  * @group FileBackend
  * @group medium
+ *
+ * @covers FileBackend
+ * 
+ * @covers CopyFileOp
+ * @covers CreateFileOp
+ * @covers DeleteFileOp
+ * @covers DescribeFileOp
+ * @covers FSFile
+ * @covers FSFileBackend
+ * @covers FSFileBackendDirList
+ * @covers FSFileBackendFileList
+ * @covers FSFileBackendList
+ * @covers FSFileOpHandle
+ * @covers FileBackendDBRepoWrapper
+ * @covers FileBackendError
+ * @covers FileBackendGroup
+ * @covers FileBackendMultiWrite
+ * @covers FileBackendStore
+ * @covers FileBackendStoreOpHandle
+ * @covers FileBackendStoreShardDirIterator
+ * @covers FileBackendStoreShardFileIterator
+ * @covers FileBackendStoreShardListIterator
+ * @covers FileJournal
+ * @covers FileOp
+ * @covers FileOpBatch
+ * @covers HTTPFileStreamer
+ * @covers LockManagerGroup
+ * @covers MemoryFileBackend
+ * @covers MoveFileOp
+ * @covers MySqlLockManager
+ * @covers NullFileJournal
+ * @covers NullFileOp
+ * @covers StoreFileOp
+ * @covers TempFSFile
+ *
+ * @covers FSLockManager
+ * @covers LockManager
+ * @covers NullLockManager
  */
 class FileBackendTest extends MediaWikiTestCase {
 
@@ -89,7 +127,6 @@
 
/**
 * @dataProvider provider_testIsStoragePath
-* @covers FileBackend::isStoragePath
 */
public function testIsStoragePath( $path, $isStorePath ) {
$this->assertEquals( $isStorePath, FileBackend::isStoragePath( 
$path ),
@@ -114,7 +151,6 @@
 
/**
 * @dataProvider provider_testSplitStoragePath
-* @covers FileBackend::splitStoragePath
 */
public function testSplitStoragePath( $path, $res ) {
$this->assertEquals( $res, FileBackend::splitStoragePath( $path 
),
@@ -139,7 +175,6 @@
 
/**
 * @dataProvider provider_normalizeStoragePath
-* @covers FileBackend::normalizeStoragePath
 */
public function testNormalizeStoragePath( $path, $res ) {
$this->assertEquals( $res, FileBackend::normalizeStoragePath( 
$path ),
@@ -169,7 +204,6 @@
 
/**
 * @dataProvider provider_testParentStoragePath
-* @covers FileBackend::parentStoragePath
 */
public function testParentStoragePath( $path, $res ) {
$this->assertEquals( $res, FileBackend::parentStoragePath( 
$path ),
@@ -191,7 +225,6 @@
 
/**
 * @dataProvider provider_testExtensionFromPath
-* @covers FileBackend::extensionFromPath
 */
public function testExtensionFromPath( $path, $res ) {
$this->assertEquals( $res, FileBackend::extensionFromPath( 
$path ),
@@ -224,9 +257,6 @@
$this->tearDownFiles();
}
 
-   /**
-* @covers FileBackend::doOperation
-*/
private function doTestStore( $op ) {
$backendName = $this->backendClass();
 
@@ -286,7 +316,6 @@
 
/**
 * @dataProvider provider_testCopy
-* @covers FileBackend::doOperation
 */
public function testCopy( $op ) {
$this->backend = $this->singleBackend;
@@ -407,7 +436,6 @@
 
/**
 * @dataProvider provider_testMove
-* @covers FileBackend::doOperation
 */
public function testMove( $op ) {
$this->backend = $this->singleBackend;
@@ -529,7 +557,6 @@
 
/**
 * @dataProvider provider_testDelete
-* @covers FileBackend::doOperation
 */
public function testDelete( $op, $withSource, $okStatus ) {
$this->backend = $this->singleBackend;
@@ -621,7 +648,6 @@
 
/**
 * @dataProvider provider_testDescribe
-* @c

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix @covers for (non-integration) parser tests

2017-02-26 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340062 )

Change subject: Fix @covers for (non-integration) parser tests
..

Fix @covers for (non-integration) parser tests

You have to allow tests to cover private Parser methods that they
execute. Private methods will never have separate tests.

Change-Id: Ic842e2be4675f505dc26d1d3e1dd9000401df46c
---
M tests/phpunit/includes/parser/ParserMethodsTest.php
M tests/phpunit/includes/parser/ParserPreloadTest.php
M tests/phpunit/includes/parser/TagHooksTest.php
3 files changed, 50 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/62/340062/1

diff --git a/tests/phpunit/includes/parser/ParserMethodsTest.php 
b/tests/phpunit/includes/parser/ParserMethodsTest.php
index 5e00384..ae58d1c 100644
--- a/tests/phpunit/includes/parser/ParserMethodsTest.php
+++ b/tests/phpunit/includes/parser/ParserMethodsTest.php
@@ -2,8 +2,8 @@
 
 /**
  * @group Database
+ * @covers Parser
  */
-
 class ParserMethodsTest extends MediaWikiLangTestCase {
 
public static function providePreSaveTransform() {
@@ -19,7 +19,6 @@
 
/**
 * @dataProvider providePreSaveTransform
-* @covers Parser::preSaveTransform
 */
public function testPreSaveTransform( $text, $expected ) {
global $wgParser;
@@ -63,7 +62,6 @@
 
/**
 * @dataProvider provideStripOuterParagraph
-* @covers Parser::stripOuterParagraph
 */
public function testStripOuterParagraph( $text, $expected ) {
$this->assertEquals( $expected, Parser::stripOuterParagraph( 
$text ) );
@@ -73,7 +71,6 @@
 * @expectedException MWException
 * @expectedExceptionMessage Parser state cleared while parsing.
 *  Did you call Parser::parse recursively?
-* @covers Parser::lock
 */
public function testRecursiveParse() {
global $wgParser;
@@ -90,9 +87,6 @@
return 'bar';
}
 
-   /**
-* @covers Parser::callParserFunction
-*/
public function testCallParserFunction() {
global $wgParser;
 
@@ -111,7 +105,7 @@
}
 
/**
-* @covers Parser::parse
+* @covers Parser
 * @covers ParserOutput::getSections
 */
public function testGetSections() {
@@ -155,8 +149,6 @@
 
/**
 * @dataProvider provideNormalizeLinkUrl
-* @covers Parser::normalizeLinkUrl
-* @covers Parser::normalizeUrlComponent
 */
public function testNormalizeLinkUrl( $explanation, $url, $expected ) {
$this->assertEquals( $expected, Parser::normalizeLinkUrl( $url 
), $explanation );
diff --git a/tests/phpunit/includes/parser/ParserPreloadTest.php 
b/tests/phpunit/includes/parser/ParserPreloadTest.php
index 414ab8f..7707395 100644
--- a/tests/phpunit/includes/parser/ParserPreloadTest.php
+++ b/tests/phpunit/includes/parser/ParserPreloadTest.php
@@ -2,6 +2,30 @@
 /**
  * Basic tests for Parser::getPreloadText
  * @author Antoine Musso
+ *
+ * @covers Parser
+ * @covers StripState
+ *
+ * @covers Preprocessor_DOM
+ * @covers PPDStack
+ * @covers PPDStackElement
+ * @covers PPDPart
+ * @covers PPFrame_DOM
+ * @covers PPTemplateFrame_DOM
+ * @covers PPCustomFrame_DOM
+ * @covers PPNode_DOM
+ *
+ * @covers Preprocessor_Hash
+ * @covers PPDStack_Hash
+ * @covers PPDStackElement_Hash
+ * @covers PPDPart_Hash
+ * @covers PPFrame_Hash
+ * @covers PPTemplateFrame_Hash
+ * @covers PPCustomFrame_Hash
+ * @covers PPNode_Hash_Tree
+ * @covers PPNode_Hash_Text
+ * @covers PPNode_Hash_Array
+ * @covers PPNode_Hash_Attr
  */
 class ParserPreloadTest extends MediaWikiTestCase {
/**
@@ -37,16 +61,10 @@
unset( $this->title );
}
 
-   /**
-* @covers Parser::getPreloadText
-*/
public function testPreloadSimpleText() {
$this->assertPreloaded( 'simple', 'simple' );
}
 
-   /**
-* @covers Parser::getPreloadText
-*/
public function testPreloadedPreIsUnstripped() {
$this->assertPreloaded(
'monospaced',
@@ -55,9 +73,6 @@
);
}
 
-   /**
-* @covers Parser::getPreloadText
-*/
public function testPreloadedNowikiIsUnstripped() {
$this->assertPreloaded(
'[[Dummy title]]',
diff --git a/tests/phpunit/includes/parser/TagHooksTest.php 
b/tests/phpunit/includes/parser/TagHooksTest.php
index 2a4a4ec..12936ee 100644
--- a/tests/phpunit/includes/parser/TagHooksTest.php
+++ b/tests/phpunit/includes/parser/TagHooksTest.php
@@ -3,6 +3,30 @@
 /**
  * @group Database
  * @group Parser
+ *
+ * @covers Parser
+ * @covers StripState
+ *
+ * @covers Preprocessor_DOM
+ * @covers PPDStack
+ * @covers PPDStac

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Fix @covers for BalancerTest

2017-02-26 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/340061 )

Change subject: Fix @covers for BalancerTest
..

Fix @covers for BalancerTest

This test is intended to cover the whole file, not just one method.

Change-Id: Ice800ce467e030e8264db96e19feadf9b68afb9a
---
M tests/phpunit/includes/tidy/BalancerTest.php
1 file changed, 6 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/61/340061/1

diff --git a/tests/phpunit/includes/tidy/BalancerTest.php 
b/tests/phpunit/includes/tidy/BalancerTest.php
index d93181c..8a4f662 100644
--- a/tests/phpunit/includes/tidy/BalancerTest.php
+++ b/tests/phpunit/includes/tidy/BalancerTest.php
@@ -13,7 +13,12 @@
}
 
/**
-* @covers MediaWiki\Tidy\Balancer::balance
+* @covers MediaWiki\Tidy\Balancer
+* @covers MediaWiki\Tidy\BalanceSets
+* @covers MediaWiki\Tidy\BalanceElement
+* @covers MediaWiki\Tidy\BalanceStack
+* @covers MediaWiki\Tidy\BalanceMarker
+* @covers MediaWiki\Tidy\BalanceActiveFormattingElements
 * @dataProvider provideBalancerTests
 */
public function testBalancer( $description, $input, $expected, $useTidy 
) {

-- 
To view, visit https://gerrit.wikimedia.org/r/340061
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ice800ce467e030e8264db96e19feadf9b68afb9a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/vendor[master]: Add wikimedia/remex-html

2017-02-23 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/339600 )

Change subject: Add wikimedia/remex-html
..

Add wikimedia/remex-html

For I900155b7dd199b0ae2a3b9cdb6db5136fc4f35a8

Change-Id: I864f31d9afdffdde49bfd39f07a0fb7f4df5c5d9
---
M composer.json
M composer.lock
M composer/autoload_classmap.php
M composer/autoload_psr4.php
M composer/autoload_static.php
M composer/installed.json
A wikimedia/remex-html/LICENSE
A wikimedia/remex-html/README.md
A wikimedia/remex-html/src/DOM/DOMBuilder.php
A wikimedia/remex-html/src/DOM/DOMFormatter.php
A wikimedia/remex-html/src/DOM/DOMSerializer.php
A wikimedia/remex-html/src/DOM/DOMUtils.php
A wikimedia/remex-html/src/GenerateDataFiles.php
A wikimedia/remex-html/src/HTMLData.php
A wikimedia/remex-html/src/PropGuard.php
A wikimedia/remex-html/src/Serializer/AbstractSerializer.php
A wikimedia/remex-html/src/Serializer/DepurateFormatter.php
A wikimedia/remex-html/src/Serializer/FastFormatter.php
A wikimedia/remex-html/src/Serializer/Formatter.php
A wikimedia/remex-html/src/Serializer/HtmlFormatter.php
A wikimedia/remex-html/src/Serializer/Serializer.php
A wikimedia/remex-html/src/Serializer/SerializerError.php
A wikimedia/remex-html/src/Serializer/SerializerNode.php
A wikimedia/remex-html/src/Serializer/TestFormatter.php
A wikimedia/remex-html/src/Tokenizer/Attribute.php
A wikimedia/remex-html/src/Tokenizer/Attributes.php
A wikimedia/remex-html/src/Tokenizer/LazyAttributes.php
A wikimedia/remex-html/src/Tokenizer/PlainAttributes.php
A wikimedia/remex-html/src/Tokenizer/TestTokenHandler.php
A wikimedia/remex-html/src/Tokenizer/TokenGenerator.php
A wikimedia/remex-html/src/Tokenizer/TokenGeneratorHandler.php
A wikimedia/remex-html/src/Tokenizer/TokenHandler.php
A wikimedia/remex-html/src/Tokenizer/TokenSerializer.php
A wikimedia/remex-html/src/Tokenizer/Tokenizer.php
A wikimedia/remex-html/src/Tokenizer/TokenizerError.php
A wikimedia/remex-html/src/TreeBuilder/ActiveFormattingElements.php
A wikimedia/remex-html/src/TreeBuilder/AfterAfterBody.php
A wikimedia/remex-html/src/TreeBuilder/AfterAfterFrameset.php
A wikimedia/remex-html/src/TreeBuilder/AfterBody.php
A wikimedia/remex-html/src/TreeBuilder/AfterFrameset.php
A wikimedia/remex-html/src/TreeBuilder/AfterHead.php
A wikimedia/remex-html/src/TreeBuilder/BeforeHead.php
A wikimedia/remex-html/src/TreeBuilder/BeforeHtml.php
A wikimedia/remex-html/src/TreeBuilder/CachingStack.php
A wikimedia/remex-html/src/TreeBuilder/DestructTracer.php
A wikimedia/remex-html/src/TreeBuilder/DestructTracerNode.php
A wikimedia/remex-html/src/TreeBuilder/DispatchTracer.php
A wikimedia/remex-html/src/TreeBuilder/Dispatcher.php
A wikimedia/remex-html/src/TreeBuilder/Element.php
A wikimedia/remex-html/src/TreeBuilder/ForeignAttributes.php
A wikimedia/remex-html/src/TreeBuilder/FormattingElement.php
A wikimedia/remex-html/src/TreeBuilder/InBody.php
A wikimedia/remex-html/src/TreeBuilder/InCaption.php
A wikimedia/remex-html/src/TreeBuilder/InCell.php
A wikimedia/remex-html/src/TreeBuilder/InColumnGroup.php
A wikimedia/remex-html/src/TreeBuilder/InForeignContent.php
A wikimedia/remex-html/src/TreeBuilder/InFrameset.php
A wikimedia/remex-html/src/TreeBuilder/InHead.php
A wikimedia/remex-html/src/TreeBuilder/InHeadNoscript.php
A wikimedia/remex-html/src/TreeBuilder/InPre.php
A wikimedia/remex-html/src/TreeBuilder/InRow.php
A wikimedia/remex-html/src/TreeBuilder/InSelect.php
A wikimedia/remex-html/src/TreeBuilder/InSelectInTable.php
A wikimedia/remex-html/src/TreeBuilder/InTable.php
A wikimedia/remex-html/src/TreeBuilder/InTableBody.php
A wikimedia/remex-html/src/TreeBuilder/InTableText.php
A wikimedia/remex-html/src/TreeBuilder/InTemplate.php
A wikimedia/remex-html/src/TreeBuilder/InTextarea.php
A wikimedia/remex-html/src/TreeBuilder/Initial.php
A wikimedia/remex-html/src/TreeBuilder/InsertionMode.php
A wikimedia/remex-html/src/TreeBuilder/Marker.php
A wikimedia/remex-html/src/TreeBuilder/SimpleStack.php
A wikimedia/remex-html/src/TreeBuilder/Stack.php
A wikimedia/remex-html/src/TreeBuilder/TemplateModeStack.php
A wikimedia/remex-html/src/TreeBuilder/Text.php
A wikimedia/remex-html/src/TreeBuilder/TreeBuilder.php
A wikimedia/remex-html/src/TreeBuilder/TreeBuilderError.php
A wikimedia/remex-html/src/TreeBuilder/TreeHandler.php
A wikimedia/remex-html/src/TreeBuilder/TreeMutationTracer.php
79 files changed, 15,218 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vendor 
refs/changes/00/339600/1


-- 
To view, visit https://gerrit.wikimedia.org/r/339600
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I864f31d9afdffdde49bfd39f07a0fb7f4df5c5d9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vendor
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
h

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Explain "fast not elegant" in README.md

2017-02-23 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/339596 )

Change subject: Explain "fast not elegant" in README.md
..

Explain "fast not elegant" in README.md

Per Subbu's suggestion

Change-Id: I41b6c63ca937e2eda295dd0aa03f3ad04292ba25
---
M README.md
1 file changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/96/339596/1

diff --git a/README.md b/README.md
index ebc416e..cf12074 100644
--- a/README.md
+++ b/README.md
@@ -6,7 +6,9 @@
 RemexHtml aims to be:
 
 - Modular and flexible.
-- Fast, as opposed to elegant.
+- Fast, as opposed to elegant. For example, we sometimes use direct member
+  access instead of going through accessors, and manually inline some
+  performance-sensitive code.
 - Robust, aiming for O(N) worst-case performance.
 
 RemexHtml contains the following modules:

-- 
To view, visit https://gerrit.wikimedia.org/r/339596
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I41b6c63ca937e2eda295dd0aa03f3ad04292ba25
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/libs/RemexHtml
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Route PHP warnings from the handler into logstash

2017-02-20 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/338820 )

Change subject: Route PHP warnings from the handler into logstash
..

Route PHP warnings from the handler into logstash

Bug: T45086
Change-Id: If34503314d7f359c005d030329111be912a758d9
---
M wmf-config/InitialiseSettings.php
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/20/338820/1

diff --git a/wmf-config/InitialiseSettings.php 
b/wmf-config/InitialiseSettings.php
index 545c315..c492a2e 100644
--- a/wmf-config/InitialiseSettings.php
+++ b/wmf-config/InitialiseSettings.php
@@ -4672,6 +4672,8 @@
'deprecated' => 'debug',
'diff' => 'debug',
'Echo' => 'debug',
+   'error' => [ 'logstash' => false ],
+   'error-json' => 'debug',
'es-hit' => 'debug',
'EventBus' => [ 'logstash' => false ], // T150106, temporarily 
disable to logstash (bd808)
'EventLogging' => 'debug',

-- 
To view, visit https://gerrit.wikimedia.org/r/338820
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If34503314d7f359c005d030329111be912a758d9
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Log a backtrace from the culprit location if headers were al...

2017-02-19 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/338705 )

Change subject: Log a backtrace from the culprit location if headers were 
already sent
..

Log a backtrace from the culprit location if headers were already sent

Install the backtrace collector very early, so that we can get the
backtrace even if headers were sent from LocalSettings.php.

Bug: T157392
Change-Id: I9bc732b34481c95afb5362e135a87bd4302498e2
---
M autoload.php
A includes/HeaderCallback.php
M includes/Setup.php
M includes/WebResponse.php
M includes/WebStart.php
M includes/resourceloader/ResourceLoader.php
6 files changed, 71 insertions(+), 32 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/05/338705/1

diff --git a/autoload.php b/autoload.php
index b21310e..fce7891 100644
--- a/autoload.php
+++ b/autoload.php
@@ -852,6 +852,7 @@
'MediaWiki\\Auth\\UsernameAuthenticationRequest' => __DIR__ . 
'/includes/auth/UsernameAuthenticationRequest.php',
'MediaWiki\\Diff\\ComplexityException' => __DIR__ . 
'/includes/diff/ComplexityException.php',
'MediaWiki\\Diff\\WordAccumulator' => __DIR__ . 
'/includes/diff/WordAccumulator.php',
+   'MediaWiki\\HeaderCallback' => __DIR__ . '/includes/HeaderCallback.php',
'MediaWiki\\Interwiki\\ClassicInterwikiLookup' => __DIR__ . 
'/includes/interwiki/ClassicInterwikiLookup.php',
'MediaWiki\\Interwiki\\InterwikiLookup' => __DIR__ . 
'/includes/interwiki/InterwikiLookup.php',
'MediaWiki\\Interwiki\\InterwikiLookupAdapter' => __DIR__ . 
'/includes/interwiki/InterwikiLookupAdapter.php',
diff --git a/includes/HeaderCallback.php b/includes/HeaderCallback.php
new file mode 100644
index 000..510cec6
--- /dev/null
+++ b/includes/HeaderCallback.php
@@ -0,0 +1,65 @@
+warning(
+   'Cookies set on {url} with 
Cache-Control "{cache-control}"', [
+   'url' => 
\WebRequest::getGlobalRequestURL(),
+   'cookies' => 
$headers['set-cookie'],
+   'cache-control' => 
$cacheControl ?: '',
+   ]
+   );
+   }
+   }
+
+   // Save a backtrace for logging in case it turns out that 
headers were sent prematurely
+   self::$headersSentException = new \Exception( 'Headers already 
sent from this point' );
+   }
+
+   /**
+* Log a warning message if headers have already been sent. This can be
+* called before flushing the output.
+*/
+   public static function warnIfHeadersSent() {
+   if ( headers_sent() && !self::$messageSent ) {
+   self::$messageSent = true;
+   \MWDebug::warning( 'Headers already sent, should send 
headers earlier than ' . wfGetCaller( 3 ) );
+   $logger = \MediaWiki\Logger\LoggerFactory::getInstance( 
'headers-sent' );
+   $logger->error( 'Warning: headers already sent', [
+   'exception' => self::$headersSentException
+   ] );
+   }
+   }
+}
diff --git a/includes/Setup.php b/includes/Setup.php
index 01ba1e8..72ed1fd 100644
--- a/includes/Setup.php
+++ b/includes/Setup.php
@@ -521,35 +521,6 @@
 // is complete.
 define( 'MW_SERVICE_BOOTSTRAP_COMPLETE', 1 );
 
-// Install a header callback to prevent caching of responses with cookies 
(T127993)
-if ( !$wgCommandLineMode ) {
-   header_register_callback( function () {
-   $headers = [];
-   foreach ( headers_list() as $header ) {
-   list( $name, $value ) = explode( ':', $header, 2 );
-   $headers[strtolower( trim( $name ) )][] = trim( $value 
);
-   }
-
-   if ( isset( $headers['set-cookie'] ) ) {
-   $cacheControl = isset( $headers['cache-control'] )
-   ? implode( ', ', $headers['cache-control'] )
-   : '';
-
-   if ( !preg_match( 
'/(?:^|,)\s*(?:private|no-cache|no-store)\s*(?:$|,)/i', $cacheControl ) ) {
-   header( 'Expires: Thu, 01 Jan 1970 00:00:00 
GMT' );
-   header( 'Cache-Control: private, max-age=0, 
s-maxage=0' );
-   MediaWiki\Logger\LoggerFactory::getInstance( 
'cache-cookies' )->warning(
-   'Cookies set on {url} with 
Cache-Control "{cache-control}"', [
-   'url' => 
WebRequest::getGlobalRequestURL(),
-   'cookies' => 
$headers['set-cookie'],
-   'cache-co

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Guard against runtime creation of undeclared properties

2017-02-16 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/338309 )

Change subject: Guard against runtime creation of undeclared properties
..

Guard against runtime creation of undeclared properties

And fix the bugs which were thus detected.

Also fix an error in TokenGeneratorHandler.

Change-Id: Ica8ebc5ea910e6093e330e90c3e58f9ae167595a
---
A src/PropGuard.php
M src/Serializer/Serializer.php
M src/Serializer/SerializerNode.php
M src/Tokenizer/Attribute.php
M src/Tokenizer/TokenGeneratorHandler.php
M src/Tokenizer/Tokenizer.php
M src/TreeBuilder/Element.php
M src/TreeBuilder/InsertionMode.php
M src/TreeBuilder/Marker.php
M src/TreeBuilder/TreeBuilder.php
M tests/phpunit/SerializerNodeDestructionTest.php
11 files changed, 81 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/09/338309/1

diff --git a/src/PropGuard.php b/src/PropGuard.php
new file mode 100644
index 000..00c3c4d
--- /dev/null
+++ b/src/PropGuard.php
@@ -0,0 +1,21 @@
+$name = $value;
+   }
+   }
+}
diff --git a/src/Serializer/Serializer.php b/src/Serializer/Serializer.php
index 65da72e..ad7ec8d 100644
--- a/src/Serializer/Serializer.php
+++ b/src/Serializer/Serializer.php
@@ -1,6 +1,7 @@
 errorCallback = $errorCallback;
}
 
+   public function __set( $name, $value ) {
+   PropGuard::set( $this, $name, $value );
+   }
+
/**
 * Get the final string. This can only be called after endDocument() is 
received.
 */
diff --git a/src/Serializer/SerializerNode.php 
b/src/Serializer/SerializerNode.php
index f19b20e..2944732 100644
--- a/src/Serializer/SerializerNode.php
+++ b/src/Serializer/SerializerNode.php
@@ -1,6 +1,7 @@
 attrs = $attrs;
$this->void = $void;
}
+
+   public function __set( $name, $value ) {
+   PropGuard::set( $this, $name, $value );
+   }
 }
diff --git a/src/Tokenizer/Attribute.php b/src/Tokenizer/Attribute.php
index b53e3c0..293b075 100644
--- a/src/Tokenizer/Attribute.php
+++ b/src/Tokenizer/Attribute.php
@@ -1,6 +1,7 @@
 localName = $localName;
$this->value = $value;
}
+
+   public function __set( $name, $value ) {
+   PropGuard::set( $this, $name, $value );
+   }
 }
diff --git a/src/Tokenizer/TokenGeneratorHandler.php 
b/src/Tokenizer/TokenGeneratorHandler.php
index 4d05b73..29f94f0 100644
--- a/src/Tokenizer/TokenGeneratorHandler.php
+++ b/src/Tokenizer/TokenGeneratorHandler.php
@@ -16,7 +16,7 @@
];
}
 
-   public function endDocument() {
+   public function endDocument( $pos ) {
$this->tokens[] = [ 'type' => 'endDocument' ];
}
 
diff --git a/src/Tokenizer/Tokenizer.php b/src/Tokenizer/Tokenizer.php
index f571aba..78b2383 100644
--- a/src/Tokenizer/Tokenizer.php
+++ b/src/Tokenizer/Tokenizer.php
@@ -2,6 +2,7 @@
 
 namespace RemexHtml\Tokenizer;
 use RemexHtml\HTMLData;
+use RemexHtml\PropGuard;
 
 /**
  * HTML 5 tokenizer
@@ -67,6 +68,8 @@
protected $ignoreErrors;
protected $ignoreCharRefs;
protected $ignoreNulls;
+   protected $skipPreprocess;
+   protected $appropriateEndTag;
protected $listener;
protected $state;
protected $preprocessed;
@@ -110,6 +113,10 @@
$this->skipPreprocess = !empty( $options['skipPreprocess'] );
}
 
+   public function __set( $name, $value ) {
+   PropGuard::set( $this, $name, $value );
+   }
+
public function setEnableCdataCallback( $cb ) {
$this->enableCdataCallback = $cb;
}
diff --git a/src/TreeBuilder/Element.php b/src/TreeBuilder/Element.php
index c571a12..56d0ff5 100644
--- a/src/TreeBuilder/Element.php
+++ b/src/TreeBuilder/Element.php
@@ -2,6 +2,7 @@
 
 namespace RemexHtml\TreeBuilder;
 use RemexHtml\HTMLData;
+use RemexHtml\PropGuard;
 use RemexHtml\Tokenizer\Attributes;
 
 /**
@@ -54,6 +55,12 @@
 * Internal to CachingStack. A link in the scope list.
 */
public $nextScope;
+
+   /**
+* Internal to CachingStack and SimpleStack. The current stack index, or
+* null if the element is not in the stack.
+*/
+   public $stackIndex;
 
/**
 * Internal to ActiveFormattingElements.
@@ -118,6 +125,10 @@
$this->attrs = $attrs;
}
 
+   public function __set( $name, $value ) {
+   PropGuard::set( $this, $name, $value );
+   }
+
/**
 * Is the element a MathML text integration point?
 *
diff --git a/src/TreeBuilder/InsertionMode.php 
b/src/TreeBuilder/InsertionMode.php
index 9b4e6b8..5d4cd08 100644
--- a/src/TreeBuilder/InsertionMode.php
+++ b/src/TreeBuilder/InsertionMode.php
@@ -1,6 +1,7 @@
 dispatcher = $dispatcher;
}
 
+   public function __set( $name, $val

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Add Serializer::getLastChild()

2017-02-15 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/338051 )

Change subject: Add Serializer::getLastChild()
..

Add Serializer::getLastChild()

Change-Id: Icd5df8c83930849cb3438393f99ff88654d187a3
---
M src/Serializer/Serializer.php
1 file changed, 19 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/51/338051/1

diff --git a/src/Serializer/Serializer.php b/src/Serializer/Serializer.php
index 51e66b9..65da72e 100644
--- a/src/Serializer/Serializer.php
+++ b/src/Serializer/Serializer.php
@@ -72,6 +72,8 @@
 
/**
 * Get the root SerializerNode.
+*
+* @return SerializerNode
 */
public function getRootNode() {
return $this->root;
@@ -79,9 +81,25 @@
 
/**
 * Get the parent SerializerNode of a given SerializerNode
+*
+* @param SerializerNode $node
+* @return SerializerNode
 */
public function getParentNode( SerializerNode $node ) {
return $this->nodes[$node->parentId];
+   }
+
+   /**
+* Get the last child of a given SerializerNode
+*
+* @param SerializerNode $node
+* @return SerializerNode|string|null
+*/
+   public function getLastChild( SerializerNode $node ) {
+   $children = $node->children;
+   $lastChildIndex = count( $children ) - 1;
+   $lastChild = $lastChildIndex >= 0 ? $children[$lastChildIndex] 
: null;
+   return $lastChild;
}
 
public function startDocument( $fragmentNamespace, $fragmentName ) {
@@ -110,7 +128,7 @@
$this->nextNodeId = 0;
}
 
-   private function interpretPlacement( $preposition, $refElement ) {
+   protected function interpretPlacement( $preposition, $refElement ) {
if ( $preposition === TreeBuilder::ROOT ) {
return [ $this->root, null ];
}

-- 
To view, visit https://gerrit.wikimedia.org/r/338051
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Icd5df8c83930849cb3438393f99ff88654d187a3
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/libs/RemexHtml
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Fix CachingStack::dump

2017-02-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/337773 )

Change subject: Fix CachingStack::dump
..

Fix CachingStack::dump

Change-Id: I8d84da0440b05e47226252c975dfbc4a68aa5d54
---
M src/TreeBuilder/CachingStack.php
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/73/337773/1

diff --git a/src/TreeBuilder/CachingStack.php b/src/TreeBuilder/CachingStack.php
index d501f47..74112e1 100644
--- a/src/TreeBuilder/CachingStack.php
+++ b/src/TreeBuilder/CachingStack.php
@@ -406,10 +406,10 @@
public function dump() {
return parent::dump() .
$this->scopeDump( self::SCOPE_DEFAULT, 'In scope' ) .
-   $this->scopeDump( self::SCOPE_DEFAULT, 'In list scope' 
) .
-   $this->scopeDump( self::SCOPE_DEFAULT, 'In button 
scope' ) .
-   $this->scopeDump( self::SCOPE_DEFAULT, 'In table scope' 
) .
-   $this->scopeDump( self::SCOPE_DEFAULT, 'In select 
scope' ) . "\n";
+   $this->scopeDump( self::SCOPE_LIST, 'In list scope' ) .
+   $this->scopeDump( self::SCOPE_BUTTON, 'In button scope' 
) .
+   $this->scopeDump( self::SCOPE_TABLE, 'In table scope' ) 
.
+   $this->scopeDump( self::SCOPE_SELECT, 'In select scope' 
) . "\n";
}
 
private function scopeDump( $scopeId, $scopeName ) {

-- 
To view, visit https://gerrit.wikimedia.org/r/337773
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8d84da0440b05e47226252c975dfbc4a68aa5d54
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/libs/RemexHtml
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Composer updates: ext-mbstring, add gitattributes

2017-02-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/337772 )

Change subject: Composer updates: ext-mbstring, add gitattributes
..

Composer updates: ext-mbstring, add gitattributes

Change-Id: I66a8da97f78d3012c95056953bd557c13ea18290
---
A .gitattributes
M composer.json
2 files changed, 12 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/72/337772/1

diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 000..412d375
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,11 @@
+/.gitattributes export-ignore
+/.gitignore export-ignore
+/.gitreview export-ignore
+/Doxyfile export-ignore
+/composer.json export-ignore
+/doc export-ignore
+/phpcs.xml export-ignore
+/phpunit.xml.dist export-ignore
+/tests export-ignore
+/bin export-ignore
+/src/FuzzTest export-ignore
diff --git a/composer.json b/composer.json
index 78a2ca3..356c9dd 100644
--- a/composer.json
+++ b/composer.json
@@ -9,6 +9,7 @@
}
],
"require": {
+   "ext-mbstring": "*",
"wikimedia/utfnormal": "1.*"
},
"require-dev": {

-- 
To view, visit https://gerrit.wikimedia.org/r/337772
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I66a8da97f78d3012c95056953bd557c13ea18290
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/libs/RemexHtml
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Some Serializer changes to support extending it

2017-02-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/337774 )

Change subject: Some Serializer changes to support extending it
..

Some Serializer changes to support extending it

RemexCompatMunger (in MediaWiki core) makes some changes to the tree
mutation event stream before passing the events on to Serializer. Add
some facilities for this.

* Add getParentNode() and getRootNode()
* Add SerializerNode::$snData for user data
* Allow Serializer methods to take a SerializerNode for $refElement

Change-Id: I913eb3a8f430b2510bd3e3a9753af5ea4018e555
---
M src/Serializer/Serializer.php
M src/Serializer/SerializerNode.php
2 files changed, 50 insertions(+), 24 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/74/337774/1

diff --git a/src/Serializer/Serializer.php b/src/Serializer/Serializer.php
index a31adeb..51e66b9 100644
--- a/src/Serializer/Serializer.php
+++ b/src/Serializer/Serializer.php
@@ -70,6 +70,20 @@
return $this->result;
}
 
+   /**
+* Get the root SerializerNode.
+*/
+   public function getRootNode() {
+   return $this->root;
+   }
+
+   /**
+* Get the parent SerializerNode of a given SerializerNode
+*/
+   public function getParentNode( SerializerNode $node ) {
+   return $this->nodes[$node->parentId];
+   }
+
public function startDocument( $fragmentNamespace, $fragmentName ) {
$this->root = new SerializerNode( 0, 0, '', '', new 
PlainAttributes, false );
$this->nodes = [ $this->root ];
@@ -96,16 +110,28 @@
$this->nextNodeId = 0;
}
 
+   private function interpretPlacement( $preposition, $refElement ) {
+   if ( $preposition === TreeBuilder::ROOT ) {
+   return [ $this->root, null ];
+   }
+   if ( $refElement instanceof Element ) {
+   $refNode = $refElement->userData;
+   } elseif ( $refElement instanceof SerializerNode ) {
+   $refNode = $refElement;
+   } else {
+   throw new SerializerError( "Invalid type of ref 
element" );
+   }
+   if ( $preposition === TreeBuilder::BEFORE ) {
+   return [ $this->nodes[$refNode->parentId], $refNode ];
+   } else {
+   return [ $refNode, $refNode ];
+   }
+   }
+
public function characters( $preposition, $refElement, $text, $start, 
$length,
$sourceStart, $sourceLength
) {
-   if ( $preposition === TreeBuilder::ROOT ) {
-   $parent = $this->root;
-   } elseif ( $preposition === TreeBuilder::BEFORE ) {
-   $parent = $this->nodes[$refElement->userData->parentId];
-   } else {
-   $parent = $refElement->userData;
-   }
+   list( $parent, $refNode ) = $this->interpretPlacement( 
$preposition, $refElement );
$encoded = (string)$this->formatter->characters( $parent, 
$text, $start, $length );
 
$children =& $parent->children;
@@ -114,7 +140,6 @@
 
if ( $preposition === TreeBuilder::BEFORE ) {
// Insert before element
-   $refNode = $refElement->userData;
if ( $lastChild !== $refNode ) {
$refIndex = array_search( $refNode, $children, 
true );
throw new SerializerError( "invalid insert 
position $refIndex/$lastChildIndex" );
@@ -131,16 +156,20 @@
}
}
 
+   /**
+* Insert an element
+*
+* @param integer $preposition
+* @param Element|SerializerNode|null $refElement
+* @param Element $element
+* @param bool $void
+* @param integer $sourceStart
+* @param integer $sourceLength
+*/
public function insertElement( $preposition, $refElement, Element 
$element, $void,
$sourceStart, $sourceLength
) {
-   if ( $preposition === TreeBuilder::ROOT ) {
-   $parent = $this->root;
-   } elseif ( $preposition === TreeBuilder::BEFORE ) {
-   $parent = $this->nodes[$refElement->userData->parentId];
-   } else {
-   $parent = $refElement->userData;
-   }
+   list( $parent, $refNode ) = $this->interpretPlacement( 
$preposition, $refElement );
$children =& $parent->children;
$lastChildIndex = count( $children ) - 1;
$lastChild = $lastChildIndex >= 0 ? $children[$lastChildIndex] 
: null;
@@ -168,7 +197,6 @@
 
if

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: [WIP] RemexHtml tidy driver with p-wrapping

2017-02-14 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/337771 )

Change subject: [WIP] RemexHtml tidy driver with p-wrapping
..

[WIP] RemexHtml tidy driver with p-wrapping

Some tests still fail. Requires non-existent version of RemexHtml.

Change-Id: I900155b7dd199b0ae2a3b9cdb6db5136fc4f35a8
---
M autoload.php
M composer.json
A includes/tidy/RemexCompatFormatter.php
A includes/tidy/RemexCompatMunger.php
A includes/tidy/RemexDriver.php
A includes/tidy/RemexMungerData.php
A tests/phpunit/includes/tidy/RemexDriverTest.php
7 files changed, 720 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/71/337771/1

diff --git a/autoload.php b/autoload.php
index 0e719ae..595b6c0 100644
--- a/autoload.php
+++ b/autoload.php
@@ -914,6 +914,10 @@
'MediaWiki\\Tidy\\RaggettInternalHHVM' => __DIR__ . 
'/includes/tidy/RaggettInternalHHVM.php',
'MediaWiki\\Tidy\\RaggettInternalPHP' => __DIR__ . 
'/includes/tidy/RaggettInternalPHP.php',
'MediaWiki\\Tidy\\RaggettWrapper' => __DIR__ . 
'/includes/tidy/RaggettWrapper.php',
+   'MediaWiki\\Tidy\\RemexDriver' => __DIR__ . 
'/includes/tidy/RemexDriver.php',
+   'MediaWiki\\Tidy\\RemexCompatMunger' => __DIR__ . 
'/includes/tidy/RemexCompatMunger.php',
+   'MediaWiki\\Tidy\\RemexCompatFormatter' => __DIR__ . 
'/includes/tidy/RemexCompatFormatter.php',
+   'MediaWiki\\Tidy\\RemexMungerData' => __DIR__ . 
'/includes/tidy/RemexMungerData.php',
'MediaWiki\\Tidy\\TidyDriverBase' => __DIR__ . 
'/includes/tidy/TidyDriverBase.php',
'MediaWiki\\Widget\\ComplexNamespaceInputWidget' => __DIR__ . 
'/includes/widget/ComplexNamespaceInputWidget.php',
'MediaWiki\\Widget\\ComplexTitleInputWidget' => __DIR__ . 
'/includes/widget/ComplexTitleInputWidget.php',
diff --git a/composer.json b/composer.json
index d41492e..a3d5546 100644
--- a/composer.json
+++ b/composer.json
@@ -38,6 +38,7 @@
"wikimedia/ip-set": "1.1.0",
"wikimedia/php-session-serializer": "1.0.4",
"wikimedia/relpath": "1.0.3",
+   "wikimedia/remex-html": "~1.0",
"wikimedia/running-stat": "1.1.0",
"wikimedia/scoped-callback": "1.0.0",
"wikimedia/utfnormal": "1.1.0",
diff --git a/includes/tidy/RemexCompatFormatter.php 
b/includes/tidy/RemexCompatFormatter.php
new file mode 100644
index 000..2183432
--- /dev/null
+++ b/includes/tidy/RemexCompatFormatter.php
@@ -0,0 +1,68 @@
+ true,
+   'p' => true,
+   'tr' => true,
+   ];
+
+   public function __construct( $options = [] ) {
+   parent::__construct( $options );
+   $this->attributeEscapes["\xc2\xa0"] = ' ';
+   unset( $this->attributeEscapes["&"] );
+   $this->textEscapes["\xc2\xa0"] = ' ';
+   unset( $this->textEscapes["&"] );
+   }
+
+   public function startDocument( $fragmentNamespace, $fragmentName ) {
+   return '';
+   }
+
+   public function element( SerializerNode $parent, SerializerNode $node, 
$contents ) {
+   $data = $node->snData;
+   if ( $data->isPWrapper ) {
+   if ( !$data->isDisabledPWrapper && !$data->isBlank ) {
+   return "$contents";
+   } else {
+   return $contents;
+   }
+   }
+
+   $name = $node->name;
+   $attrs = $node->attrs;
+   if ( isset( self::$markedEmptyElements[$name] ) && 
$attrs->count() === 0 ) {
+   if ( strspn( $contents, "\t\n\f\r " ) === strlen( 
$contents ) ) {
+   return "<{$name} 
class=\"mw-empty-elt\">$contents";
+   }
+   }
+
+   $s = "<$name";
+   foreach ( $attrs->getValues() as $attrName => $attrValue ) {
+   $encValue = strtr( $attrValue, $this->attributeEscapes 
);
+   $s .= " $attrName=\"$encValue\"";
+   }
+   if ( $node->namespace === HTMLData::NS_HTML && isset( 
$this->voidElements[$name] ) ) {
+   $s .= ' />';
+   return $s;
+   }
+
+   $s .= '>';
+   if ( $node->namespace === HTMLData::NS_HTML
+   && isset( $contents[0] ) && $contents[0] === "\n"
+   && isset( $this->prefixLfElements[$name] )
+   ) {
+   $s .= "\n$contents";
+   } else {
+   $s .= "$contents";
+   }
+   return $s;
+   }
+}
diff --git a/includes/tidy/RemexCompatMunger.php 
b/includes/tidy/RemexCompatMunger.php
new file mode 100644
index 000..3783d61
--- /dev/null
+++ b/i

[MediaWiki-commits] [Gerrit] mediawiki...ParsoidBatchAPI[master]: Do not generate responsive images for iconThumb

2017-02-07 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/336569 )

Change subject: Do not generate responsive images for iconThumb
..

Do not generate responsive images for iconThumb

Should fix the notice "Undefined index: width" which is presumably due
to Linker::processResponsiveImages() being called when
makeTransformOptions() fails to fetch the handler and so returns an
empty array.

Bug: T138987
Change-Id: Ib661c0a562c5ae68d5591df362558956e982b6a1
---
M includes/ApiParsoidBatch.php
1 file changed, 9 insertions(+), 6 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ParsoidBatchAPI 
refs/changes/69/336569/1

diff --git a/includes/ApiParsoidBatch.php b/includes/ApiParsoidBatch.php
index 5f38ecd..7fb456f 100644
--- a/includes/ApiParsoidBatch.php
+++ b/includes/ApiParsoidBatch.php
@@ -287,12 +287,15 @@
if ( $mto->isError() ) {
$result['thumberror'] = $mto->toText();
} else {
-   // Do srcset scaling
-   Linker::processResponsiveImages( $file, $mto, 
$txopts );
-   if ( count( $mto->responsiveUrls ) ) {
-   $result['responsiveUrls'] = array();
-   foreach ( $mto->responsiveUrls as 
$density => $url ) {
-   
$result['responsiveUrls'][$density] = wfExpandUrl( $url, PROTO_CURRENT );
+   if ( $txopts ) {
+   // Do srcset scaling
+   Linker::processResponsiveImages( $file, 
$mto, $txopts );
+   if ( count( $mto->responsiveUrls ) ) {
+   $result['responsiveUrls'] = 
array();
+   foreach ( $mto->responsiveUrls 
as $density => $url ) {
+   
$result['responsiveUrls'][$density] = wfExpandUrl(
+   $url, 
PROTO_CURRENT );
+   }
}
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/336569
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib661c0a562c5ae68d5591df362558956e982b6a1
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ParsoidBatchAPI
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Add XML infoset coercion to DOMBuilder

2017-02-01 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/335596 )

Change subject: Add XML infoset coercion to DOMBuilder
..

Add XML infoset coercion to DOMBuilder

* In DOMBuilder, encode names that are invalid in XML
* For performance, trigger encoding names only when an exception is caught.
* Invert the encoding in TestFormatter, and optionally in HtmlFormatter
* Re-enable tests that now pass

Change-Id: Iddfa497e07636693baa280575b086f46adefe002
---
M src/DOM/DOMBuilder.php
M src/GenerateDataFiles.php
M src/HTMLData.php
M src/Serializer/HtmlFormatter.php
M src/Serializer/TestFormatter.php
M tests/phpunit/TreeBuilderTest.php
6 files changed, 238 insertions(+), 34 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/96/335596/1

diff --git a/src/DOM/DOMBuilder.php b/src/DOM/DOMBuilder.php
index 5519e17..e4de294 100644
--- a/src/DOM/DOMBuilder.php
+++ b/src/DOM/DOMBuilder.php
@@ -10,14 +10,15 @@
  * A TreeHandler which constructs a DOMDocument
  */
 class DOMBuilder implements TreeHandler {
-   private $doc;
-   private $errorCallback;
-   private $isFragment;
-
public $doctypeName;
public $public;
public $system;
public $quirks;
+
+   private $doc;
+   private $errorCallback;
+   private $isFragment;
+   private $coerced;
 
/**
 * @param callable|null $errorCallback A function which is called on 
parse errors
@@ -45,6 +46,16 @@
}
}
 
+   /**
+* Returns true if the document was coerced due to libxml limitations. 
We
+* follow HTML 5.1 § 8.2.7 "Coercing an HTML DOM into an infoset".
+*
+* @return bool
+*/
+   public function isCoerced() {
+   return $this->coerced;
+   }
+
public function startDocument( $fragmentNamespace, $fragmentName ) {
$impl = new \DOMImplementation;
$this->isFragment = $fragmentNamespace !== null;
@@ -53,9 +64,10 @@
 
private function createDocument( $doctypeName = null, $public = null, 
$system = null ) {
$impl = new \DOMImplementation;
-   if ( $doctypeName === null
-   || $doctypeName === '' // libxml limitation, causes 
test failures
-   ) {
+   if ( $doctypeName === '' ) {
+   $this->coerced = true;
+   $doc = $impl->createDocument( null, null );
+   } elseif ( $doctypeName === null ) {
$doc = $impl->createDocument( null, null );
} else {
$doctype = $impl->createDocumentType( $doctypeName, 
$public, $system );
@@ -82,10 +94,31 @@
$parent->insertBefore( $node, $refNode );
}
 
+   /**
+* Replace unsupported characters with a code of the form U123456.
+*
+* @param string $name
+* @return string
+*/
+   private function coerceName( $name ) {
+   $coercedName = DOMUtils::coerceName( $name );
+   if ( $name !== $coercedName ) {
+   $this->coerced = true;
+   }
+   return $coercedName;
+   }
+
private function createNode( Element $element ) {
-   $node = $this->doc->createElementNS(
-   $element->namespace,
-   $element->name );
+   try {
+   $node = $this->doc->createElementNS(
+   $element->namespace,
+   $element->name );
+   } catch ( \DOMException $e ) {
+   // Attempt to escape the name so that it is more 
acceptable
+   $node = $this->doc->createElementNS(
+   $element->namespace,
+   $this->coerceName( $element->name ) );
+   }
 
foreach ( $element->attrs->getObjects() as $attr ) {
if ( $attr->namespaceURI === null
@@ -98,12 +131,26 @@
// way can't be discovered via hasAttribute() 
or hasAttributeNS().
$attrNode = $this->doc->createAttribute( 
$attr->localName );
$attrNode->value = $attr->value;
-   $node->setAttributeNodeNS( $attrNode );
+   try {
+   $node->setAttributeNodeNS( $attrNode );
+   } catch ( \DOMException $e ) {
+   $node->setAttributeNS(
+   $attr->namespaceURI,
+   $this->coerceName( 
$attr->qualifiedName ),
+  

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Fix phpcs test.php rule

2017-02-01 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/335581 )

Change subject: Fix phpcs test.php rule
..

Fix phpcs test.php rule

The rule to exclude test.php from phpcs turns out to exclude any file
with test.php in its name, case insensitively. This actually matches a
lot of files. Fix it, and fix the files with phpcs errors.

Change-Id: Ifc4831127fd26d442cb3c48ad22ba7988e0fd7d6
---
M phpcs.xml
M src/FuzzTest/FuzzTest.php
M tests/phpunit/SerializerNodeDestructionTest.php
M tests/phpunit/TokenizerTest.php
M tests/phpunit/TreeBuilderTest.php
5 files changed, 11 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/81/335581/1

diff --git a/phpcs.xml b/phpcs.xml
index 3a1e7cf..ee9bda8 100644
--- a/phpcs.xml
+++ b/phpcs.xml
@@ -9,7 +9,7 @@
.


-   vendor
-   HTMLData.php
-   test.php
+   /vendor
+   /src/HTMLData.php
+   /bin/test.php
 
diff --git a/src/FuzzTest/FuzzTest.php b/src/FuzzTest/FuzzTest.php
index 99339c2..05b6df4 100644
--- a/src/FuzzTest/FuzzTest.php
+++ b/src/FuzzTest/FuzzTest.php
@@ -26,10 +26,12 @@
for ( $seed = 0; true; $seed++ ) {
mt_srand( $seed );
$text = $tokenSalad->next();
+   // @codingStandardsIgnoreStart
if ( @iconv( 'UTF-8', 'UTF-8', $text ) === false ) {
// Skip invalid UTF-8 tests
continue;
}
+   // @codingStandardsIgnoreEnd
 
$formatter = new Serializer\DepurateFormatter( [
'scriptingFlag' => false
diff --git a/tests/phpunit/SerializerNodeDestructionTest.php 
b/tests/phpunit/SerializerNodeDestructionTest.php
index 099cf87..077e0b4 100644
--- a/tests/phpunit/SerializerNodeDestructionTest.php
+++ b/tests/phpunit/SerializerNodeDestructionTest.php
@@ -105,7 +105,8 @@
$continue = $tokenizer->step();
}
$this->assertEquals( 1, $count );
-   while ( $tokenizer->step() );
+   while ( $tokenizer->step() ) {
+   }
$this->assertEquals( 7, $count );
}
 }
diff --git a/tests/phpunit/TokenizerTest.php b/tests/phpunit/TokenizerTest.php
index 3e4669e..5322471 100644
--- a/tests/phpunit/TokenizerTest.php
+++ b/tests/phpunit/TokenizerTest.php
@@ -33,7 +33,7 @@
continue;
}
foreach ( $testData['tests'] as $test ) {
-   $states = isset( $test['initialStates'] ) ? 
$test['initialStates']  : ['data state'];
+   $states = isset( $test['initialStates'] ) ? 
$test['initialStates']  : [ 'data state' ];
$input = $test['input'];
$output = $test['output'];
$appropriateEndTag = isset( 
$test['lastStartTag'] ) ? $test['lastStartTag'] : null;
@@ -67,7 +67,7 @@
 */
private function unescape( $value ) {
if ( is_array( $value ) ) {
-   return array_map( array( $this, 'unescape' ), $value );
+   return array_map( [ $this, 'unescape' ], $value );
} elseif ( is_string( $value ) ) {
return preg_replace_callback( '/u([0-9a-fA-F]{4})/',
function ( $m ) {
@@ -132,7 +132,7 @@
}
 
/** @dataProvider provider */
-   public function testIgnoreErrors(  $state, $appropriateEndTag, $input, 
$expected ) {
+   public function testIgnoreErrors( $state, $appropriateEndTag, $input, 
$expected ) {
$handler = new TestTokenHandler();
$tokenizer = new Tokenizer( $handler, $input, [ 'ignoreErrors' 
=> true ] );
$tokenizer->execute( [
diff --git a/tests/phpunit/TreeBuilderTest.php 
b/tests/phpunit/TreeBuilderTest.php
index bb4cc1a..63fbf6f 100644
--- a/tests/phpunit/TreeBuilderTest.php
+++ b/tests/phpunit/TreeBuilderTest.php
@@ -48,7 +48,6 @@
return $this->provider( 'dom' );
}
 
-
private function provider( $type ) {
$testFiles = [];
foreach ( self::$testDirs as $testDir ) {
@@ -136,7 +135,7 @@
) {
continue;
}
-   
+
$tests[] = $test;
}
return $tests;

-- 
To view, visit https://gerrit.wikimedia.org/r/335581
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifc4831127fd26d442cb3c48ad22ba7988e0fd7d6
Gerrit-PatchSet: 1
Gerrit-Project: me

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Improve DOMBuilder and add tests

2017-01-31 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/335367 )

Change subject: Improve DOMBuilder and add tests
..

Improve DOMBuilder and add tests

* Move DOM-related classes to a separate namespace
* Introduce DOMSerializer which is a TreeHandler interchangeable with
  Serializer, allowing comparative testing
* Make the Formatter implementations also be able to serialize DOM
  subtrees. This allows DOM serialization to share the non-static
  member data such as void element lists with the tree mutation stream
  serializer.
* Run the html5lib tests on the DOM serializer. This requires
  blacklisting a few tests due to validation done in libxml2.

Change-Id: I4fc50a155f1f94c9f6bfdd888d0d2aebfc43637d
---
M bin/test.php
A src/DOM/DOMBuilder.php
A src/DOM/DOMFormatter.php
A src/DOM/DOMSerializer.php
A src/Serializer/AbstractSerializer.php
M src/Serializer/HtmlFormatter.php
M src/Serializer/Serializer.php
M src/Serializer/TestFormatter.php
D src/TreeBuilder/DOMBuilder.php
M tests/phpunit/TreeBuilderTest.php
10 files changed, 664 insertions(+), 179 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/67/335367/1

diff --git a/bin/test.php b/bin/test.php
index 3708a35..f0dfbea 100755
--- a/bin/test.php
+++ b/bin/test.php
@@ -7,6 +7,7 @@
 
 require __DIR__ . '/../vendor/autoload.php';
 
+use RemexHtml\DOM;
 use RemexHtml\Tokenizer;
 use RemexHtml\TreeBuilder;
 use RemexHtml\Serializer;
@@ -41,7 +42,7 @@
 function reserialize( $text ) {
$handler = new Tokenizer\TokenSerializer;
$tokenizer = new Tokenizer\Tokenizer( $handler, $text, [] );
-   $tokenizer->execute();
+   $tokenizer->execute( $GLOBALS['executeOptions'] );
print $handler->getOutput() . "\n";
foreach ( $handler->getErrors() as $error ) {
print "Error at {$error[1]}: {$error[0]}\n";
@@ -77,10 +78,7 @@
$dispatcher = new TreeBuilder\Dispatcher( $treeBuilder );
$dispatchTracer = new TreeBuilder\DispatchTracer( $text, $dispatcher, 
$traceCallback );
$tokenizer = new Tokenizer\Tokenizer( $dispatchTracer, $text, [] );
-   $tokenizer->execute( [
-   // 'fragmentNamespace' => \RemexHtml\HTMLData::NS_HTML,
-   // 'fragmentName' => 'html'
-   ] );
+   $tokenizer->execute( $GLOBALS['executeOptions'] );
 
print $serializer->getResult() . "\n";
 }
@@ -95,10 +93,7 @@
$dispatcher = new TreeBuilder\Dispatcher( $treeBuilder );
$dispatchTracer = new TreeBuilder\DispatchTracer( $text, $dispatcher, 
$traceCallback );
$tokenizer = new Tokenizer\Tokenizer( $dispatchTracer, $text, [] );
-   $tokenizer->execute( [
-   // 'fragmentNamespace' => \RemexHtml\HTMLData::NS_HTML,
-   // 'fragmentName' => 'html'
-   ] );
+   $tokenizer->execute( $GLOBALS['executeOptions'] );
 }
 
 function tidy( $text ) {
@@ -110,7 +105,48 @@
$treeBuilder = new TreeBuilder\TreeBuilder( $serializer, [] );
$dispatcher = new TreeBuilder\Dispatcher( $treeBuilder );
$tokenizer = new Tokenizer\Tokenizer( $dispatcher, $text, 
$GLOBALS['tokenizerOptions'] );
-   $tokenizer->execute();
+   $tokenizer->execute( $GLOBALS['executeOptions'] );
+   print $serializer->getResult() . "\n";
+}
+
+function test( $text ) {
+   $error = function ( $msg, $pos ) {
+   print "  *  [$pos] $msg\n";
+   };
+   $formatter = new Serializer\TestFormatter;
+   $serializer = new Serializer\Serializer( $formatter, $error );
+   $treeBuilder = new TreeBuilder\TreeBuilder( $serializer, [] );
+   $dispatcher = new TreeBuilder\Dispatcher( $treeBuilder );
+   $tokenizer = new Tokenizer\Tokenizer( $dispatcher, $text, 
$GLOBALS['tokenizerOptions'] );
+   $tokenizer->execute( $GLOBALS['executeOptions'] );
+   print $serializer->getResult() . "\n";
+}
+
+function tidyViaDOM( $text ) {
+   $error = function ( $msg, $pos ) {
+   print "  *  [$pos] $msg\n";
+   };
+   $formatter = new Serializer\HtmlFormatter;
+   $domBuilder = new DOM\DOMBuilder( $error );
+   $serializer = new DOM\DOMSerializer( $domBuilder, $formatter );
+   $treeBuilder = new TreeBuilder\TreeBuilder( $serializer, [] );
+   $dispatcher = new TreeBuilder\Dispatcher( $treeBuilder );
+   $tokenizer = new Tokenizer\Tokenizer( $dispatcher, $text, [] );
+   $tokenizer->execute( $GLOBALS['executeOptions'] );
+   print $serializer->getResult() . "\n";
+}
+
+function testViaDOM( $text ) {
+   $error = function ( $msg, $pos ) {
+   print "  *  [$pos] $msg\n";
+   };
+   $formatter = new Serializer\TestFormatter;
+   $domBuilder = new DOM\DOMBuilder( $error );
+   $serializer = new DOM\DOMSerializer( $domBuilder, $formatter );
+   $treeBuilder = new TreeBuilder\TreeBuilder( $serializer, [] );
+ 

[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Remove XhtmlFormatter and Parser

2017-01-23 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/333824 )

Change subject: Remove XhtmlFormatter and Parser
..

Remove XhtmlFormatter and Parser

They were not fully baked, and are not necessary for present purposes.

Change-Id: I4bb2c989ca131436a52f35df061b85c66091ec6c
---
M bin/test.php
D src/Serializer/XhtmlFormatter.php
D src/TreeBuilder/Parser.php
3 files changed, 5 insertions(+), 134 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/24/333824/1

diff --git a/bin/test.php b/bin/test.php
index e8a95ce..3708a35 100755
--- a/bin/test.php
+++ b/bin/test.php
@@ -66,17 +66,6 @@
reserializeState( $text, Tokenizer\Tokenizer::STATE_RCDATA, 'xmp' );
 }
 
-function traceDispatch( $text ) {
-   TreeBuilder\Parser::parseDocument( $text, [ 'traceDispatch' => true ] );
-}
-
-function traceDOM( $text ) {
-   TreeBuilder\Parser::parseDocument( $text, [
-   'traceTreeMutation' => true,
-   'traceDispatch' => true,
-   ] );
-}
-
 function trace( $text ) {
$traceCallback = function ( $msg ) {
print "$msg\n";
@@ -110,25 +99,6 @@
// 'fragmentNamespace' => \RemexHtml\HTMLData::NS_HTML,
// 'fragmentName' => 'html'
] );
-}
-
-function tidyBodyViaDOM( $text ) {
-   $docText = "\n$text";
-   $doc = TreeBuilder\Parser::parseDocument( $docText, [] );
-   $body = $doc->getElementsByTagName( 'body' )->item( 0 );
-   foreach ( $body->childNodes as $node ) {
-   print $doc->saveHTML( $node );
-   }
-   print "\n";
-}
-
-function tidyViaDOM( $text ) {
-   $doc = TreeBuilder\Parser::parseDocument( $text, [
-   'treeBuilder' => [
-   'scopeCache' => true,
-   ]
-   ] );
-   print $doc->saveHTML() . "\n";
 }
 
 function tidy( $text ) {
@@ -178,12 +148,11 @@
 
 function benchmarkDOM( $text ) {
$time = -microtime( true );
-   $dom = TreeBuilder\Parser::parseDocument( $text, [
-   'treeBuilder' => [
-   'ignoreErrors' => true,
-   ],
-   'tokenizer' => $GLOBALS['tokenizerOptions']
-   ] );
+   $domBuilder = new TreeBuilder\DOMBuilder;
+   $treeBuilder = new TreeBuilder\TreeBuilder( $domBuilder, [ 
'ignoreErrors' => true ] );
+   $dispatcher = new TreeBuilder\Dispatcher( $treeBuilder );
+   $tokenizer = new Tokenizer\Tokenizer( $dispatcher, $text, 
$GLOBALS['tokenizerOptions'] );
+   $tokenizer->execute();
$time += microtime( true );
print "$time\n";
 }
diff --git a/src/Serializer/XhtmlFormatter.php 
b/src/Serializer/XhtmlFormatter.php
deleted file mode 100644
index 8fc4f42..000
--- a/src/Serializer/XhtmlFormatter.php
+++ /dev/null
@@ -1,48 +0,0 @@
-\n";
-   }
-
-   public function characters( SerializerNode $parent, $text, $start, 
$length ) {
-   $text = substr( $text, $start, $length );
-   return strtr( $text, [
-   '<' => '<',
-   '>' => '>',
-   '&' => '&' ] );
-   }
-
-   public function element( SerializerNode $parent, SerializerNode $node, 
$contents ) {
-   $name = $node->name;
-   $ret = "<$name";
-   foreach ( $node->attrs->getValues() as $attrName => $value ) {
-   $ret .= " $attrName=\"" .
-   strtr( $value, [
-   '"' => '"',
-   '&' => '&',
-   ] ) . "\"";
-   }
-   if ( $contents === null ) {
-   $ret .= " />";
-   } elseif ( isset( $contents[0] ) && $contents[0] === "\n"
-   && in_array( $name, [ 'pre', 'textarea', 'listing' ] )
-   ) {
-   $ret .= ">\n$contents";
-   } else {
-   $ret .= ">$contents";
-   }
-   return $ret;
-   }
-
-   public function comment( SerializerNode $parent, $text ) {
-   return "";
-   }
-
-   public function doctype( $name, $public, $system ) {
-   return '';
-   }
-}
diff --git a/src/TreeBuilder/Parser.php b/src/TreeBuilder/Parser.php
deleted file mode 100644
index 871d203..000
--- a/src/TreeBuilder/Parser.php
+++ /dev/null
@@ -1,50 +0,0 @@
- [],
-   'tokenizer' => [],
-   'domBuilder' => [],
-   'traceDispatch' => false,
-   'traceTreeMutation' => false,
-   ];
-
-   $this->domBuilder = new DOMBuilder( $options['domBuilder'] );
-   if ( $options['traceTreeMutation'] ) {
-   $treeHandler = new TreeMutationTracer( 
$this-

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Protect -{...}- variant constructs in images.

2016-12-20 Thread Tim Starling (Code Review)
Tim Starling has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/327042 )

Change subject: Protect -{...}- variant constructs in images.
..


Protect -{...}- variant constructs in images.

A protected version of explode is factored out as
`StringUtils::delimiterExplode`, since it will be used in follow-up
patches in this series.  The `delimiterExplode` implementation creates
an intermediate array of the exploded results, which is reasonable as
the number of image options is small; but since an Iterator is
returned the implementation can be upgraded in the future (at the cost
of additional complexity) to avoid this.  The additional code in that
case would be similar to ExplodeIterator.

Bug: T146305
Change-Id: I1327685e9e8c07ef476dceaa6f6dae4ba40989ef
---
M includes/libs/StringUtils.php
M includes/parser/Parser.php
M tests/parser/parserTests.txt
3 files changed, 76 insertions(+), 6 deletions(-)

Approvals:
  Tim Starling: Verified; Looks good to me, approved



diff --git a/includes/libs/StringUtils.php b/includes/libs/StringUtils.php
index 6b10c09..26f3c4a 100644
--- a/includes/libs/StringUtils.php
+++ b/includes/libs/StringUtils.php
@@ -55,6 +55,59 @@
}
 
/**
+* Explode a string, but ignore any instances of the separator inside
+* the given start and end delimiters, which may optionally nest.
+* The delimiters are literal strings, not regular expressions.
+* @param string $startDelim Start delimiter
+* @param string $endDelim End delimiter
+* @param string $separator Separator string for the explode.
+* @param string $subject Subject string to explode.
+* @param bool $nested True iff the delimiters are allowed to nest.
+* @return ArrayIterator
+*/
+   static function delimiterExplode( $startDelim, $endDelim, $separator,
+   $subject, $nested = false ) {
+   $inputPos = 0;
+   $lastPos = 0;
+   $depth = 0;
+   $encStart = preg_quote( $startDelim, '!' );
+   $encEnd = preg_quote( $endDelim, '!' );
+   $encSep = preg_quote( $separator, '!' );
+   $len = strlen( $subject );
+   $m = [];
+   $exploded = [];
+   while (
+   $inputPos < $len &&
+   preg_match(
+   "!$encStart|$encEnd|$encSep!S", $subject, $m,
+   PREG_OFFSET_CAPTURE, $inputPos
+   )
+   ) {
+   $match = $m[0][0];
+   $matchPos = $m[0][1];
+   $inputPos = $matchPos + strlen( $match );
+   if ( $match === $separator ) {
+   if ( $depth === 0 ) {
+   $exploded[] = substr(
+   $subject, $lastPos, $matchPos - 
$lastPos
+   );
+   $lastPos = $inputPos;
+   }
+   } elseif ( $match === $startDelim ) {
+   if ( $depth === 0 || $nested ) {
+   $depth++;
+   }
+   } else {
+   $depth--;
+   }
+   }
+   $exploded[] = substr( $subject, $lastPos );
+   // This method could be rewritten in the future to avoid 
creating an
+   // intermediate array, since the return type is just an 
iterator.
+   return new ArrayIterator( $exploded );
+   }
+
+   /**
 * Perform an operation equivalent to `preg_replace()`
 *
 * Matches this code:
diff --git a/includes/parser/Parser.php b/includes/parser/Parser.php
index 7418547..8f9830c 100644
--- a/includes/parser/Parser.php
+++ b/includes/parser/Parser.php
@@ -5150,7 +5150,10 @@
#  * bottom
#  * text-bottom
 
-   $parts = StringUtils::explode( "|", $options );
+   # Protect LanguageConverter markup when splitting into parts
+   $parts = StringUtils::delimiterExplode(
+   '-{', '}-', '|', $options, true /* allow nesting */
+   );
 
# Give extensions a chance to select the file revision for us
$options = [];
diff --git a/tests/parser/parserTests.txt b/tests/parser/parserTests.txt
index 505bc2d..edcc2c4 100644
--- a/tests/parser/parserTests.txt
+++ b/tests/parser/parserTests.txt
@@ -20617,16 +20617,30 @@
 
 !! end
 
-# FIXME: This test is currently broken in the PHP parser (bug 52661)
 !! test
-Don't break image parsing if language converter markup is in the caption.
+T146305: Don't break image parsing if

[MediaWiki-commits] [Gerrit] integration/config[master]: Clarify fab reload prompt

2016-12-19 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/328334 )

Change subject: Clarify fab reload prompt
..

Clarify fab reload prompt

Change-Id: Ica548822f08e4402611c8660023b87df75d7b07c
---
M fabfile.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/config 
refs/changes/34/328334/1

diff --git a/fabfile.py b/fabfile.py
index 64814cb..8cb49b7 100644
--- a/fabfile.py
+++ b/fabfile.py
@@ -27,7 +27,7 @@
 sudo('git remote update')
 sudo('git --no-pager log -p HEAD..origin/master zuul')
 if confirm('Does the diff look good?') and confirm(
-'Log your reload in #wikimedia-releng (e.g. "!log Reloading' +
+'Did you log your reload in #wikimedia-releng (e.g. "!log 
Reloading' +
 ' Zuul to deploy [hash]")'):
 sudo('git rebase')
 sudo('/etc/init.d/zuul reload')

-- 
To view, visit https://gerrit.wikimedia.org/r/328334
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ica548822f08e4402611c8660023b87df75d7b07c
Gerrit-PatchSet: 1
Gerrit-Project: integration/config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] integration/config[master]: Restore "Configure jobs for mediawiki/libs/RemexHtml"

2016-12-19 Thread Tim Starling (Code Review)
Hello Legoktm, jenkins-bot,

I'd like you to do a code review.  Please visit

https://gerrit.wikimedia.org/r/328332

to review the following change.


Change subject: Restore "Configure jobs for mediawiki/libs/RemexHtml"
..

Restore "Configure jobs for mediawiki/libs/RemexHtml"

This reverts commit 7cb35174a7b0ad7db7a49d7b1efd16ddd690ca7e.

Change-Id: I0b2c84523502bfc0e2940c34ba6f96f9208a9000
---
M zuul/layout.yaml
1 file changed, 7 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/config 
refs/changes/32/328332/1

diff --git a/zuul/layout.yaml b/zuul/layout.yaml
index d9edbed..7f4387e 100644
--- a/zuul/layout.yaml
+++ b/zuul/layout.yaml
@@ -8841,6 +8841,13 @@
   #- doxygen-publish
   - phpunit-coverage-publish
 
+  - name: mediawiki/libs/RemexHtml
+template:
+ - name: composer-test-package55
+postmerge:
+  - doxygen-publish
+  - phpunit-coverage-publish
+
   - name: mediawiki/libs/ScopedCallback
 template:
  - name: composer-test-package55

-- 
To view, visit https://gerrit.wikimedia.org/r/328332
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0b2c84523502bfc0e2940c34ba6f96f9208a9000
Gerrit-PatchSet: 1
Gerrit-Project: integration/config
Gerrit-Branch: master
Gerrit-Owner: Tim Starling 
Gerrit-Reviewer: Legoktm 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits


[MediaWiki-commits] [Gerrit] mediawiki...RemexHtml[master]: Fixes for old PCRE and PHP 5

2016-12-19 Thread Tim Starling (Code Review)
Tim Starling has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/328331 )

Change subject: Fixes for old PCRE and PHP 5
..

Fixes for old PCRE and PHP 5

* In the script data regex, replace the possessive quantifier "*+"
  on the inner loop with a plain "*" quantifier. This works around the
  infinite loop bug in PCRE 8.34 which was the reason for the version
  check. The possessive quantifier was only there to make the regex
  easier to reason about. I've confirmed that it appears to still have
  O(N) performance with a non-possessive quantifier.
* Fix unnecessary namespace importation which failed on PHP 5.

Change-Id: I012c034c1657ab805a94c443d187e0e85249632a
---
M bin/test.php
M src/Tokenizer/Tokenizer.php
M tests/phpunit/TreeBuilderTest.php
3 files changed, 10 insertions(+), 14 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/libs/RemexHtml 
refs/changes/31/328331/1

diff --git a/bin/test.php b/bin/test.php
index 31c4b7f..e8a95ce 100755
--- a/bin/test.php
+++ b/bin/test.php
@@ -7,7 +7,6 @@
 
 require __DIR__ . '/../vendor/autoload.php';
 
-use RemexHtml;
 use RemexHtml\Tokenizer;
 use RemexHtml\TreeBuilder;
 use RemexHtml\Serializer;
@@ -90,7 +89,7 @@
$dispatchTracer = new TreeBuilder\DispatchTracer( $text, $dispatcher, 
$traceCallback );
$tokenizer = new Tokenizer\Tokenizer( $dispatchTracer, $text, [] );
$tokenizer->execute( [
-   // 'fragmentNamespace' => RemexHtml\HTMLData::NS_HTML,
+   // 'fragmentNamespace' => \RemexHtml\HTMLData::NS_HTML,
// 'fragmentName' => 'html'
] );
 
@@ -108,7 +107,7 @@
$dispatchTracer = new TreeBuilder\DispatchTracer( $text, $dispatcher, 
$traceCallback );
$tokenizer = new Tokenizer\Tokenizer( $dispatchTracer, $text, [] );
$tokenizer->execute( [
-   // 'fragmentNamespace' => RemexHtml\HTMLData::NS_HTML,
+   // 'fragmentNamespace' => \RemexHtml\HTMLData::NS_HTML,
// 'fragmentName' => 'html'
] );
 }
@@ -149,7 +148,10 @@
$time = -microtime( true );
$handler = new NullHandler;
$tokenizer = new Tokenizer\Tokenizer( $handler, $text, 
$GLOBALS['tokenizerOptions'] );
-   $tokenizer->execute();
+   $tokenizer->execute( [
+   //'state' => Tokenizer\Tokenizer::STATE_SCRIPT_DATA,
+   //'appropriateEndTag' => 'script'
+   ] );
$time += microtime( true );
print "$time\n";
 }
diff --git a/src/Tokenizer/Tokenizer.php b/src/Tokenizer/Tokenizer.php
index 18586c8..f571aba 100644
--- a/src/Tokenizer/Tokenizer.php
+++ b/src/Tokenizer/Tokenizer.php
@@ -108,12 +108,6 @@
$this->ignoreCharRefs = !empty( $options['ignoreCharRefs'] );
$this->ignoreNulls = !empty( $options['ignoreNulls'] );
$this->skipPreprocess = !empty( $options['skipPreprocess'] );
-
-   $version = explode( ' ', PCRE_VERSION );
-   if ( version_compare( $version[0], '8.36', '<' ) ) {
-   throw new TokenizerError( 'The script data regex 
requires PCRE 8.36 or later.' );
-   // Possibly 8.35 would work, but 8.34 fails the unit 
tests.
-   }
}
 
public function setEnableCdataCallback( $cb ) {
@@ -1329,6 +1323,7 @@
$this->pos = $this->length;
return self::STATE_EOF;
}
+
$re = <<]
)
)
-   )*+
+   )*
 
 
# Consume the comment close which 
exited the inner loop, if any
diff --git a/tests/phpunit/TreeBuilderTest.php 
b/tests/phpunit/TreeBuilderTest.php
index fd30251..24052b0 100644
--- a/tests/phpunit/TreeBuilderTest.php
+++ b/tests/phpunit/TreeBuilderTest.php
@@ -3,7 +3,6 @@
 namespace RemexHtml\TreeBuilder;
 use RemexHtml\HTMLData;
 use RemexHtml\Tokenizer;
-use RemexHtml\TreeBuilder;
 use RemexHtml\Serializer;
 
 class TreeBuilderTest extends \PHPUnit_Framework_TestCase {
@@ -167,10 +166,10 @@
}
$formatter = new Serializer\TestFormatter;
$serializer = new Serializer\Serializer( $formatter );
-   $treeBuilder = new TreeBuilder\TreeBuilder( $serializer, [
+   $treeBuilder = new TreeBuilder( $serializer, [
'scriptingFlag' => $params['scripting']
] );
-   $dispatcher = new TreeBuilder\Dispatcher( $treeBuilder );
+   $dispatcher = new Dispatcher( $treeBuilder );
$tokenizer = new Tokenizer\Tokenizer( $dispatcher, 
$params['data'], [] );
 
$tokenizerOptions = [];

-- 
To view, visit https://gerrit.wikimedia.org/r/328331
To unsu

  1   2   3   4   5   6   7   8   9   10   >