jenkins-bot has submitted this change and it was merged. (
https://gerrit.wikimedia.org/r/382725 )
Change subject: Optimize StatementsField for performance and readability
......................................................................
Optimize StatementsField for performance and readability
* This patch fixes all details I found while reviewing the parent
patch I880808d.
* I'm renaming quite a lot variables, class properties, and such. For
example, instead of passing a vague "definitions" array around, I
call it consistently "searchIndexDataFormatters", because that's
what it contains: formatter callbacks.
* The rewrite of StatementsField::getFieldData will become critical
pretty fast (we expect), when the number of properties to index
grows. Instead of a nested loop (plus the loop hidden in
getByPropertyId) it's a single loop and an array key lookup now.
Bug: T175199
Change-Id: Ib52a5fdf8e28cc17bb76757f0baa57b62aa14803
---
M docs/options.wiki
M lib/includes/DataTypeDefinitions.php
M repo/Wikibase.hooks.php
M repo/WikibaseRepo.datatypes.php
M repo/includes/Content/ItemHandler.php
M repo/includes/Content/PropertyHandler.php
M repo/includes/Search/Elastic/Fields/DescriptionsProviderFieldDefinitions.php
M repo/includes/Search/Elastic/Fields/ItemFieldDefinitions.php
M repo/includes/Search/Elastic/Fields/LabelsProviderFieldDefinitons.php
M repo/includes/Search/Elastic/Fields/PropertyFieldDefinitions.php
M repo/includes/Search/Elastic/Fields/StatementProviderFieldDefinitions.php
M repo/includes/Search/Elastic/Fields/StatementsField.php
M repo/includes/WikibaseRepo.php
M repo/tests/phpunit/includes/Content/ItemContentTest.php
M repo/tests/phpunit/includes/Content/ItemHandlerTest.php
M repo/tests/phpunit/includes/Search/Elastic/Fields/StatementsFieldTest.php
16 files changed, 137 insertions(+), 158 deletions(-)
Approvals:
Ladsgroup: Looks good to me, approved
Lucas Werkmeister (WMDE): Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/docs/options.wiki b/docs/options.wiki
index e17c651..1f97998 100644
--- a/docs/options.wiki
+++ b/docs/options.wiki
@@ -5,6 +5,7 @@
As usual, the extension is configured in MediaWiki's LocalSettings.php file.
However, Wikibase settings are placed in associative arrays,
<code>$wgWBRepoSettings</code> and <code>$wgWBClientSettings</code>
respectively, instead of individual global variables. So, if the setting
<code>foo</code> is described below, you would need to use
<code>$wgWBRepoSettings['foo']</code> or
<code>$wgWBClientSettings['foo']</code> in LocalSettings.php.
== Common Settings ==
+
=== Basic Settings ===
;entityNamespaces: Defines which kind of entity is managed in which namespace.
It is given as an associative array mapping entity types such as
<code>'item'</code> to namespace IDs. Mapping must be done for each type of
entity that should be supported.
@@ -45,6 +46,7 @@
;localClientDatabases: An array of locally accessible client databases, for
use by the <code>dispatchChanges.php</code> script. This setting determines to
which wikis changes are pushed directly. It must be given either as an
associative array, mapping global site IDs to logical database names, or, of
the database names are the same as the site IDs, as a list of databases. The
default is an empty array, indicating no local client databases.
=== Expert Settings ===
+
;dispatchBatchChunkFactor: Chunk factor used internally by the
<code>dispatchChanges.php</code> script. The default is 3. If most clients are
not interested in most changes, this factor can be raised to lower the number
of database queries needed to fetch a batch of changes.
;idBlacklist: A list of IDs to reserve and skip for new entities. IDs are
given as integers, the blacklist applies to all types of entities.
:'''Note:''' This may change in the future to allow separate blacklists for
different kinds of entities.
@@ -66,10 +68,12 @@
:;prefixSearchProfiles: Loaded from
<code>config/EntityPrefixSearchProfiles.php</code>, does not need to be defined
manually. (Cirrus)
:;defaultPrefixRescoreProfile: name of the rescoring profile to use for prefix
search. The profile should be defined in
<code>config/ElasticSearchRescoreProfiles.php</code>. (Cirrus)
:;rescoreProfiles: Loaded from
<code>config/ElasticSearchRescoreProfiles.php</code>, does not have to be
defined manually. (Cirrus).
-;searchIndexProperties: Array of names of properties that would be included in
the 'properties' field of the search index. For now, only relevant when Cirrus
search is enabled.
+;searchIndexProperties: Array of properties (by ID string) that should be
included in the "statement_keywords" field of the search index. For now, only
relevant when Cirrus search is enabled.
+
== Client Settings ==
=== Basic Settings ===
+
;namespaces: List of namespaces on the client wiki that should have access to
repository items. Default: <code>array()</code> (treated as setting is not set,
ie. namespaces are enabled).
;excludeNamespaces: List of namespaces on the client wiki to disable wikibase
links, etc. for. Default: <code>array()</code>. Example: <code>array(
NS_USER_TALK )</code>.
;repoUrl: The repository's base URL, including the schema (protocol) and
domain; This URL can be protocol-relative. Default is
<code>'//wikidata.org'</code>.
@@ -106,6 +110,7 @@
;propertyOrderUrl: URL to use for retrieving the property order used for
sorting properties by property ID. Will be ignored if set to null.
=== Expert Settings ===
+
;injectRecentChanges: Whether changes on the repository should be injected
into this wiki's recent changes table, so they show up on watchlists, etc.
Requires the <code>dispatchChanges.php</code> script to run, and this wiki to
be listed in the <code>localClientDatabases</code> setting on the repository.
;showExternalRecentChanges: Whether changes on the repository should be
displayed on Special:RecentChanges, Special:Watchlist, etc on the client wiki.
In contrast to <code>injectRecentChanges</code>, this setting just removes the
changes from the user interface. The default is <code>false</code>. This is
intended to temporarily prevent external changes from showing in order to find
or fix some issue on a live site.
;sendEchoNotification: If true, allows users on the client wiki to get a
notification when a page they created is connected to a repo item. This
requires the Echo extension.
diff --git a/lib/includes/DataTypeDefinitions.php
b/lib/includes/DataTypeDefinitions.php
index cc6cc94..0632ed7 100644
--- a/lib/includes/DataTypeDefinitions.php
+++ b/lib/includes/DataTypeDefinitions.php
@@ -318,8 +318,8 @@
* Get data formatters for search indexing for each type.
* @return callable[] List of callbacks, with keys having "VT:"
prefixes.
*/
- public function getIndexDataFormatters() {
- return $this->getMapForDefinitionField(
'search-index-data-formatter' );
+ public function getSearchIndexDataFormatterCallbacks() {
+ return $this->getMapForDefinitionField(
'search-index-data-formatter-callback' );
}
}
diff --git a/repo/Wikibase.hooks.php b/repo/Wikibase.hooks.php
index 7500e9d..aea60e9 100644
--- a/repo/Wikibase.hooks.php
+++ b/repo/Wikibase.hooks.php
@@ -985,8 +985,9 @@
}
/**
- * Add Wikibase-specific analyzer configs.
- * @param array $config
+ * Adds Wikibase-specific ElasticSearch analyzer configurations.
+ *
+ * @param array &$config
*/
public static function onCirrusSearchAnalysisConfig( &$config ) {
// Analyzer for splitting statements and extracting properties:
diff --git a/repo/WikibaseRepo.datatypes.php b/repo/WikibaseRepo.datatypes.php
index b4683f3..512ab05 100644
--- a/repo/WikibaseRepo.datatypes.php
+++ b/repo/WikibaseRepo.datatypes.php
@@ -251,9 +251,7 @@
) {
return new LiteralValueRdfBuilder( null, null );
},
- 'search-index-data-formatter' => function(
- StringValue $value
- ) {
+ 'search-index-data-formatter-callback' => function (
StringValue $value ) {
return $value->getValue();
},
],
@@ -347,9 +345,7 @@
) {
return new EntityIdRdfBuilder( $vocab, $tracker
);
},
- 'search-index-data-formatter' => function(
- EntityIdValue $value
- ) {
+ 'search-index-data-formatter-callback' => function (
EntityIdValue $value ) {
return
$value->getEntityId()->getSerialization();
},
],
diff --git a/repo/includes/Content/ItemHandler.php
b/repo/includes/Content/ItemHandler.php
index 7642472..b55eb9b 100644
--- a/repo/includes/Content/ItemHandler.php
+++ b/repo/includes/Content/ItemHandler.php
@@ -23,7 +23,7 @@
use Wikibase\Lib\Store\EntityContentDataCodec;
use Wikibase\Lib\Store\LanguageFallbackLabelDescriptionLookupFactory;
use Wikibase\Lib\Store\SiteLinkStore;
-use Wikibase\Repo\Search\Elastic\Fields\ItemFieldDefinitions;
+use Wikibase\Repo\Search\Elastic\Fields\FieldDefinitions;
use Wikibase\Repo\Validators\EntityConstraintProvider;
use Wikibase\Repo\Validators\ValidatorErrorLocalizer;
use Wikibase\Store\EntityIdLookup;
@@ -70,7 +70,7 @@
* @param SiteLinkStore $siteLinkStore
* @param EntityIdLookup $entityIdLookup
* @param LanguageFallbackLabelDescriptionLookupFactory
$labelLookupFactory
- * @param ItemFieldDefinitions $itemFieldDefinitions
+ * @param FieldDefinitions $itemFieldDefinitions
* @param PropertyDataTypeLookup $dataTypeLookup
* @param callable|null $legacyExportFormatDetector
*/
@@ -83,7 +83,7 @@
SiteLinkStore $siteLinkStore,
EntityIdLookup $entityIdLookup,
LanguageFallbackLabelDescriptionLookupFactory
$labelLookupFactory,
- ItemFieldDefinitions $itemFieldDefinitions,
+ FieldDefinitions $itemFieldDefinitions,
PropertyDataTypeLookup $dataTypeLookup,
$legacyExportFormatDetector = null
) {
diff --git a/repo/includes/Content/PropertyHandler.php
b/repo/includes/Content/PropertyHandler.php
index cbd1c31..2cc9b5a 100644
--- a/repo/includes/Content/PropertyHandler.php
+++ b/repo/includes/Content/PropertyHandler.php
@@ -19,7 +19,7 @@
use Wikibase\PropertyContent;
use Wikibase\PropertyInfoBuilder;
use Wikibase\Lib\Store\PropertyInfoStore;
-use Wikibase\Repo\Search\Elastic\Fields\PropertyFieldDefinitions;
+use Wikibase\Repo\Search\Elastic\Fields\FieldDefinitions;
use Wikibase\Repo\Validators\EntityConstraintProvider;
use Wikibase\Repo\Validators\ValidatorErrorLocalizer;
use Wikibase\Store\EntityIdLookup;
@@ -66,7 +66,7 @@
* @param LanguageFallbackLabelDescriptionLookupFactory
$labelLookupFactory
* @param PropertyInfoStore $infoStore
* @param PropertyInfoBuilder $propertyInfoBuilder
- * @param PropertyFieldDefinitions $propertyFieldDefinitions
+ * @param FieldDefinitions $propertyFieldDefinitions
* @param callable|null $legacyExportFormatDetector
*/
public function __construct(
@@ -79,7 +79,7 @@
LanguageFallbackLabelDescriptionLookupFactory
$labelLookupFactory,
PropertyInfoStore $infoStore,
PropertyInfoBuilder $propertyInfoBuilder,
- PropertyFieldDefinitions $propertyFieldDefinitions,
+ FieldDefinitions $propertyFieldDefinitions,
$legacyExportFormatDetector = null
) {
parent::__construct(
diff --git
a/repo/includes/Search/Elastic/Fields/DescriptionsProviderFieldDefinitions.php
b/repo/includes/Search/Elastic/Fields/DescriptionsProviderFieldDefinitions.php
index d666407..d7a9008 100644
---
a/repo/includes/Search/Elastic/Fields/DescriptionsProviderFieldDefinitions.php
+++
b/repo/includes/Search/Elastic/Fields/DescriptionsProviderFieldDefinitions.php
@@ -20,14 +20,12 @@
}
/**
- * @return array
+ * @return WikibaseIndexField[]
*/
public function getFields() {
- $fields = [];
-
- $fields['descriptions'] = new DescriptionsField(
$this->languageCodes );
-
- return $fields;
+ return [
+ 'descriptions' => new DescriptionsField(
$this->languageCodes ),
+ ];
}
}
diff --git a/repo/includes/Search/Elastic/Fields/ItemFieldDefinitions.php
b/repo/includes/Search/Elastic/Fields/ItemFieldDefinitions.php
index 4d6a6b3..2896778 100644
--- a/repo/includes/Search/Elastic/Fields/ItemFieldDefinitions.php
+++ b/repo/includes/Search/Elastic/Fields/ItemFieldDefinitions.php
@@ -1,4 +1,5 @@
<?php
+
namespace Wikibase\Repo\Search\Elastic\Fields;
/**
@@ -7,24 +8,24 @@
class ItemFieldDefinitions implements FieldDefinitions {
/**
- * @var LabelsProviderFieldDefinitions
+ * @var FieldDefinitions
*/
private $labelsProviderFieldDefinitions;
/**
- * @var DescriptionsProviderFieldDefinitions
+ * @var FieldDefinitions
*/
private $descriptionsProviderFieldDefinitions;
/**
- * @var StatementProviderFieldDefinitions
+ * @var FieldDefinitions
*/
private $statementProviderFieldDefinitions;
public function __construct(
- LabelsProviderFieldDefinitions $labelsProviderFieldDefinitions,
- DescriptionsProviderFieldDefinitions
$descriptionsProviderFieldDefinitions,
- StatementProviderFieldDefinitions
$statementProviderFieldDefinitions
+ FieldDefinitions $labelsProviderFieldDefinitions,
+ FieldDefinitions $descriptionsProviderFieldDefinitions,
+ FieldDefinitions $statementProviderFieldDefinitions
) {
$this->labelsProviderFieldDefinitions =
$labelsProviderFieldDefinitions;
$this->descriptionsProviderFieldDefinitions =
$descriptionsProviderFieldDefinitions;
@@ -35,13 +36,6 @@
* @return WikibaseIndexField[]
*/
public function getFields() {
- /*
- * Items have:
- * - labels
- * - descriptions
- * - link count
- * - statements
- */
$fields = array_merge(
$this->labelsProviderFieldDefinitions->getFields(),
$this->descriptionsProviderFieldDefinitions->getFields(),
diff --git
a/repo/includes/Search/Elastic/Fields/LabelsProviderFieldDefinitons.php
b/repo/includes/Search/Elastic/Fields/LabelsProviderFieldDefinitons.php
index 5acd525..2c0963d 100644
--- a/repo/includes/Search/Elastic/Fields/LabelsProviderFieldDefinitons.php
+++ b/repo/includes/Search/Elastic/Fields/LabelsProviderFieldDefinitons.php
@@ -1,4 +1,5 @@
<?php
+
namespace Wikibase\Repo\Search\Elastic\Fields;
/**
@@ -22,11 +23,11 @@
* @return WikibaseIndexField[]
*/
public function getFields() {
- $fields['label_count'] = new LabelCountField();
- $fields['labels'] = new LabelsField( $this->languageCodes );
- $fields['labels_all'] = new AllLabelsField();
-
- return $fields;
+ return [
+ 'label_count' => new LabelCountField(),
+ 'labels' => new LabelsField( $this->languageCodes ),
+ 'labels_all' => new AllLabelsField(),
+ ];
}
}
diff --git a/repo/includes/Search/Elastic/Fields/PropertyFieldDefinitions.php
b/repo/includes/Search/Elastic/Fields/PropertyFieldDefinitions.php
index 0db6a14..26b317a 100644
--- a/repo/includes/Search/Elastic/Fields/PropertyFieldDefinitions.php
+++ b/repo/includes/Search/Elastic/Fields/PropertyFieldDefinitions.php
@@ -1,4 +1,5 @@
<?php
+
namespace Wikibase\Repo\Search\Elastic\Fields;
/**
@@ -7,24 +8,24 @@
class PropertyFieldDefinitions implements FieldDefinitions {
/**
- * @var LabelsProviderFieldDefinitions
+ * @var FieldDefinitions
*/
private $labelsProviderFieldDefinitions;
/**
- * @var DescriptionsProviderFieldDefinitions
+ * @var FieldDefinitions
*/
private $descriptionsProviderFieldDefinitions;
/**
- * @var StatementProviderFieldDefinitions
+ * @var FieldDefinitions
*/
private $statementProviderFieldDefinitions;
public function __construct(
- LabelsProviderFieldDefinitions $labelsProviderFieldDefinitions,
- DescriptionsProviderFieldDefinitions
$descriptionsProviderFieldDefinitions,
- StatementProviderFieldDefinitions
$statementProviderFieldDefinitions
+ FieldDefinitions $labelsProviderFieldDefinitions,
+ FieldDefinitions $descriptionsProviderFieldDefinitions,
+ FieldDefinitions $statementProviderFieldDefinitions
) {
$this->labelsProviderFieldDefinitions =
$labelsProviderFieldDefinitions;
$this->descriptionsProviderFieldDefinitions =
$descriptionsProviderFieldDefinitions;
@@ -35,20 +36,11 @@
* @return WikibaseIndexField[]
*/
public function getFields() {
- /*
- * Properties have:
- * - labels
- * - descriptions
- * - statements
- * - statement count
- */
- $fields = array_merge(
+ return array_merge(
$this->labelsProviderFieldDefinitions->getFields(),
$this->descriptionsProviderFieldDefinitions->getFields(),
$this->statementProviderFieldDefinitions->getFields()
);
-
- return $fields;
}
}
diff --git
a/repo/includes/Search/Elastic/Fields/StatementProviderFieldDefinitions.php
b/repo/includes/Search/Elastic/Fields/StatementProviderFieldDefinitions.php
index 285c2d0..87599fa 100644
--- a/repo/includes/Search/Elastic/Fields/StatementProviderFieldDefinitions.php
+++ b/repo/includes/Search/Elastic/Fields/StatementProviderFieldDefinitions.php
@@ -2,8 +2,6 @@
namespace Wikibase\Repo\Search\Elastic\Fields;
-use Wikibase\Lib\DataTypeDefinitions;
-
/**
* Fields for an object that has statements.
*/
@@ -13,21 +11,20 @@
* List of properties to index.
* @var string[]
*/
- private $properties;
+ private $propertyIds;
/**
* @var callable[]
*/
- private $definitions;
+ private $searchIndexDataFormatters;
/**
- * StatementProviderFieldDefinitions constructor.
- * @param string[] $properties List of properties to index
- * @param callable[] $definitions
+ * @param string[] $propertyIds List of properties to index
+ * @param callable[] $searchIndexDataFormatters
*/
- public function __construct( array $properties, array $definitions ) {
- $this->definitions = $definitions;
- $this->properties = $properties;
+ public function __construct( array $propertyIds, array
$searchIndexDataFormatters ) {
+ $this->propertyIds = $propertyIds;
+ $this->searchIndexDataFormatters = $searchIndexDataFormatters;
}
/**
@@ -35,9 +32,13 @@
* @return WikibaseIndexField[] key is field name, value is
WikibaseIndexField
*/
public function getFields() {
- $fields['statement_keywords'] = new StatementsField(
$this->properties, $this->definitions );
- $fields['statement_count'] = new StatementCountField();
- return $fields;
+ return [
+ 'statement_keywords' => new StatementsField(
+ $this->propertyIds,
+ $this->searchIndexDataFormatters
+ ),
+ 'statement_count' => new StatementCountField(),
+ ];
}
}
diff --git a/repo/includes/Search/Elastic/Fields/StatementsField.php
b/repo/includes/Search/Elastic/Fields/StatementsField.php
index 985fbcd..9a035ec 100644
--- a/repo/includes/Search/Elastic/Fields/StatementsField.php
+++ b/repo/includes/Search/Elastic/Fields/StatementsField.php
@@ -1,12 +1,15 @@
<?php
+
namespace Wikibase\Repo\Search\Elastic\Fields;
use CirrusSearch;
+use MWException;
use SearchEngine;
+use SearchIndexField;
use SearchIndexFieldDefinition;
use Wikibase\DataModel\Entity\EntityDocument;
-use Wikibase\DataModel\Entity\PropertyId;
use Wikibase\DataModel\Snak\PropertyValueSnak;
+use Wikibase\DataModel\Statement\Statement;
use Wikibase\DataModel\Statement\StatementListProvider;
/**
@@ -23,25 +26,24 @@
const STATEMENT_SEPARATOR = '=';
/**
- * List of properties to index
- * @var string[]
+ * @var array List of properties to index, as a flipped array with the
property IDs as keys.
*/
- private $properties;
+ private $propertyIds;
/**
* @var callable[]
*/
- private $definitions;
+ private $searchIndexDataFormatters;
/**
- * StatementsField constructor.
- * @param string[] $properties
- * @param callable[] $definitions
+ * @param string[] $propertyIds
+ * @param callable[] $searchIndexDataFormatters
*/
- public function __construct( array $properties, array $definitions ) {
- $this->properties = $properties;
- $this->definitions = $definitions;
- parent::__construct( "", \SearchIndexField::INDEX_TYPE_KEYWORD
);
+ public function __construct( array $propertyIds, array
$searchIndexDataFormatters ) {
+ parent::__construct( '', SearchIndexField::INDEX_TYPE_KEYWORD );
+
+ $this->propertyIds = array_flip( $propertyIds );
+ $this->searchIndexDataFormatters = $searchIndexDataFormatters;
}
/**
@@ -50,19 +52,21 @@
* @param SearchEngine $engine
* @param string $name
*
- * @return \SearchIndexField|null Null if mapping is not supported
+ * @return SearchIndexField|null Null if mapping is not supported
*/
public function getMappingField( SearchEngine $engine, $name ) {
if ( !( $engine instanceof CirrusSearch ) ) {
// For now only Cirrus/Elastic is supported
return null;
}
+
return $this;
}
/**
* @param EntityDocument $entity
*
+ * @throws MWException
* @return mixed Get the value of the field to be indexed when a
page/document
* is indexed. This might be an array with nested data,
if the field
* is defined with nested type or an int or string for
simple field types.
@@ -71,34 +75,41 @@
if ( !( $entity instanceof StatementListProvider ) ) {
return [];
}
+
$data = [];
- foreach ( $this->properties as $property ) {
- try {
- $id = new PropertyId( $property );
- } catch ( \Exception $e ) {
- // If we couldn't resolve ID for this property,
skip it
+ /** @var Statement $statement */
+ foreach ( $entity->getStatements() as $statement ) {
+ $snak = $statement->getMainSnak();
+ if ( !( $snak instanceof PropertyValueSnak ) ) {
+ // Won't index novalue/somevalue for now
continue;
}
- foreach ( $entity->getStatements()->getByPropertyId(
$id )->getMainSnaks() as $snak ) {
- if ( !( $snak instanceof PropertyValueSnak ) ) {
- // Won't index novalue/somevalue for now
- continue;
- }
- $dataValue = $snak->getDataValue();
- if ( !isset( $this->definitions["VT:" .
$dataValue->getType()] ) ) {
- // We do not know how to format these
values
- continue;
- }
- $callback = $this->definitions["VT:" .
$dataValue->getType()];
- $value = $callback( $dataValue );
- if ( !$value ) {
- continue;
- }
- $data[] =
$snak->getPropertyId()->getSerialization() . self::STATEMENT_SEPARATOR
- . $value;
+ $propertyId =
$snak->getPropertyId()->getSerialization();
+ if ( !array_key_exists( $propertyId, $this->propertyIds
) ) {
+ continue;
}
+
+ $dataValue = $snak->getDataValue();
+ $definitionKey = 'VT:' . $dataValue->getType();
+
+ if ( !isset(
$this->searchIndexDataFormatters[$definitionKey] ) ) {
+ // We do not know how to format these values
+ continue;
+ }
+
+ $formatter =
$this->searchIndexDataFormatters[$definitionKey];
+ $value = $formatter( $dataValue );
+
+ if ( !is_string( $value ) ) {
+ throw new MWException( 'Search index data
formatter callback for "' . $definitionKey
+ . '" didn\'t return a string' );
+ } elseif ( $value === '' ) {
+ continue;
+ }
+
+ $data[] = $propertyId . self::STATEMENT_SEPARATOR .
$value;
}
return $data;
@@ -106,6 +117,7 @@
/**
* @param SearchEngine $engine
+ *
* @return array
*/
public function getMapping( SearchEngine $engine ) {
@@ -118,13 +130,13 @@
$config = [
'type' => 'keyword',
- "ignore_above" => 255
+ 'ignore_above' => 255,
];
// Subfield indexing only property names, so we could do matches
// like "property exists" without specifying the value.
$config['fields']['property'] = [
'type' => 'text',
- 'analyzer' => "extract_wb_property",
+ 'analyzer' => 'extract_wb_property',
'search_analyzer' => 'keyword',
];
diff --git a/repo/includes/WikibaseRepo.php b/repo/includes/WikibaseRepo.php
index 10a47dc..918617f 100644
--- a/repo/includes/WikibaseRepo.php
+++ b/repo/includes/WikibaseRepo.php
@@ -96,6 +96,7 @@
use Wikibase\Repo\ChangeOp\EntityChangeOpProvider;
use Wikibase\Repo\Localizer\ChangeOpDeserializationExceptionLocalizer;
use Wikibase\Repo\Search\Elastic\Fields\DescriptionsProviderFieldDefinitions;
+use Wikibase\Repo\Search\Elastic\Fields\FieldDefinitions;
use Wikibase\Repo\Search\Elastic\Fields\ItemFieldDefinitions;
use Wikibase\Repo\Search\Elastic\Fields\LabelsProviderFieldDefinitions;
use Wikibase\Repo\Search\Elastic\Fields\PropertyFieldDefinitions;
@@ -1451,14 +1452,14 @@
}
/**
- * @return LabelsProviderFieldDefinitions
+ * @return FieldDefinitions
*/
public function getLabelProviderDefinitions() {
return new LabelsProviderFieldDefinitions(
$this->getTermsLanguages()->getLanguages() );
}
/**
- * @return DescriptionsProviderFieldDefinitions
+ * @return FieldDefinitions
*/
public function getDescriptionProviderDefinitions() {
return new DescriptionsProviderFieldDefinitions(
$this->getTermsLanguages()
@@ -1466,17 +1467,17 @@
}
/**
- * @return StatementProviderFieldDefinitions
+ * @return FieldDefinitions
*/
public function getStatementProviderDefinitions() {
return new StatementProviderFieldDefinitions(
$this->settings->getSetting( 'searchIndexProperties' ),
-
$this->getDataTypeDefinitions()->getIndexDataFormatters()
+
$this->getDataTypeDefinitions()->getSearchIndexDataFormatterCallbacks()
);
}
/**
- * @return ItemFieldDefinitions
+ * @return FieldDefinitions
*/
private function getItemFieldDefinitions() {
return new ItemFieldDefinitions(
@@ -1486,7 +1487,7 @@
}
/**
- * @return PropertyFieldDefinitions
+ * @return FieldDefinitions
*/
private function getPropertyFieldDefinitions() {
return new PropertyFieldDefinitions(
diff --git a/repo/tests/phpunit/includes/Content/ItemContentTest.php
b/repo/tests/phpunit/includes/Content/ItemContentTest.php
index 6be1ee4..5d9a33a 100644
--- a/repo/tests/phpunit/includes/Content/ItemContentTest.php
+++ b/repo/tests/phpunit/includes/Content/ItemContentTest.php
@@ -27,7 +27,7 @@
use Wikibase\ItemContent;
use Wikibase\Repo\Content\EntityContentDiff;
use Wikibase\Repo\Content\ItemHandler;
-use Wikibase\Repo\Search\Elastic\Fields\ItemFieldDefinitions;
+use Wikibase\Repo\Search\Elastic\Fields\FieldDefinitions;
use Wikibase\Repo\WikibaseRepo;
/**
@@ -266,9 +266,6 @@
*/
private function getItemHandler() {
$wikibaseRepo = WikibaseRepo::getDefaultInstance();
- $itemFieldDefinitions = $this->getMockBuilder(
ItemFieldDefinitions::class )
- ->disableOriginalConstructor()
- ->getMock();
return new ItemHandler(
$wikibaseRepo->getStore()->getTermIndex(),
@@ -279,7 +276,7 @@
$wikibaseRepo->getStore()->newSiteLinkStore(),
$wikibaseRepo->getEntityIdLookup(),
$wikibaseRepo->getLanguageFallbackLabelDescriptionLookupFactory(),
- $itemFieldDefinitions,
+ $this->getMock( FieldDefinitions::class ),
$this->getPropertyDataTypeLookup()
);
}
diff --git a/repo/tests/phpunit/includes/Content/ItemHandlerTest.php
b/repo/tests/phpunit/includes/Content/ItemHandlerTest.php
index 25bf2c1..f83ff24 100644
--- a/repo/tests/phpunit/includes/Content/ItemHandlerTest.php
+++ b/repo/tests/phpunit/includes/Content/ItemHandlerTest.php
@@ -20,7 +20,7 @@
use Wikibase\EntityContent;
use Wikibase\ItemContent;
use Wikibase\Repo\Content\ItemHandler;
-use Wikibase\Repo\Search\Elastic\Fields\ItemFieldDefinitions;
+use Wikibase\Repo\Search\Elastic\Fields\FieldDefinitions;
use Wikibase\Repo\WikibaseRepo;
use Wikibase\SettingsArray;
@@ -215,9 +215,6 @@
*/
private function getItemHandlerWithMockedPropertyDataTypeLookup() {
$wikibaseRepo = WikibaseRepo::getDefaultInstance();
- $itemFieldDefinitions = $this->getMockBuilder(
ItemFieldDefinitions::class )
- ->disableOriginalConstructor()
- ->getMock();
return new ItemHandler(
$wikibaseRepo->getStore()->getTermIndex(),
@@ -228,7 +225,7 @@
$wikibaseRepo->getStore()->newSiteLinkStore(),
$wikibaseRepo->getEntityIdLookup(),
$wikibaseRepo->getLanguageFallbackLabelDescriptionLookupFactory(),
- $itemFieldDefinitions,
+ $this->getMock( FieldDefinitions::class ),
$this->getPropertyDataTypeLookup()
);
}
diff --git
a/repo/tests/phpunit/includes/Search/Elastic/Fields/StatementsFieldTest.php
b/repo/tests/phpunit/includes/Search/Elastic/Fields/StatementsFieldTest.php
index 2f8a8ef..5755352 100644
--- a/repo/tests/phpunit/includes/Search/Elastic/Fields/StatementsFieldTest.php
+++ b/repo/tests/phpunit/includes/Search/Elastic/Fields/StatementsFieldTest.php
@@ -4,16 +4,14 @@
use CirrusSearch;
use DataValues\BooleanValue;
-use DataValues\DecimalValue;
use DataValues\StringValue;
use DataValues\UnboundedQuantityValue;
use PHPUnit_Framework_TestCase;
use Wikibase\DataModel\Entity\EntityDocument;
-use Wikibase\DataModel\Entity\PropertyId;
use Wikibase\DataModel\Snak\PropertyNoValueSnak;
use Wikibase\DataModel\Snak\PropertySomeValueSnak;
use Wikibase\DataModel\Snak\PropertyValueSnak;
-use Wikibase\DataModel\Statement\StatementListProvider;
+use Wikibase\DataModel\Statement\StatementList;
use Wikibase\Repo\Search\Elastic\Fields\StatementsField;
use Wikibase\Repo\Tests\ChangeOp\StatementListProviderDummy;
use Wikibase\Repo\Tests\Rdf\RdfBuilderTestData;
@@ -24,7 +22,6 @@
*
* @group WikibaseElastic
* @group Wikibase
- *
*/
class StatementsFieldTest extends PHPUnit_Framework_TestCase {
@@ -61,55 +58,42 @@
/**
* @dataProvider statementsProvider
- * @param EntityDocument $entity
- * @param $expected
*/
- public function testStatements( EntityDocument $entity, $expected ) {
+ public function testStatements( EntityDocument $entity, array $expected
) {
if ( !class_exists( CirrusSearch::class ) ) {
$this->markTestSkipped( 'CirrusSearch needed.' );
}
- $repo = WikibaseRepo::getDefaultInstance();
- $field = new StatementsField( $this->properties,
$repo->getDataTypeDefinitions()->getIndexDataFormatters() );
+ $repo = WikibaseRepo::getDefaultInstance();
+ $field = new StatementsField( $this->properties,
$repo->getDataTypeDefinitions()->getSearchIndexDataFormatterCallbacks() );
$this->assertEquals( $expected, $field->getFieldData( $entity )
);
}
public function testFormatters() {
$formatters = [
'VT:string' => function ( StringValue $s ) {
- return "STRING:" . $s->getValue();
+ return 'STRING:' . $s->getValue();
},
-
'VT:quantity' => function ( UnboundedQuantityValue $v )
{
- return "VALUE:" . $v->getAmount();
+ return 'VALUE:' . $v->getAmount();
},
-
];
$field = new StatementsField( [ 'P123' ], $formatters );
- $snaks = [
- new PropertyValueSnak( 123, new StringValue(
'testString' ) ),
- new PropertyValueSnak( 123,
- new UnboundedQuantityValue( new DecimalValue(
456 ), "1" ) ),
- new PropertySomeValueSnak( 123 ),
- new PropertyValueSnak( 123, new StringValue(
'testString2' ) ),
- new PropertyNoValueSnak( 123 ),
- new PropertyValueSnak( 123, new BooleanValue( false ) ),
- ];
+ $statementList = new StatementList();
+ $statementList->addNewStatement( new PropertyValueSnak( 123,
new StringValue( 'testString' ) ) );
+ $statementList->addNewStatement( new PropertyValueSnak( 123,
UnboundedQuantityValue::newFromNumber( 456 ) ) );
+ $statementList->addNewStatement( new PropertySomeValueSnak( 123
) );
+ $statementList->addNewStatement( new PropertyValueSnak( 123,
new StringValue( 'testString2' ) ) );
+ $statementList->addNewStatement( new PropertyNoValueSnak( 123 )
);
+ $statementList->addNewStatement( new PropertyValueSnak( 123,
new BooleanValue( false ) ) );
- $mockList = $this->getMockBuilder( StatementListProvider::class
)->setMethods( [
- 'getByPropertyId',
- 'getMainSnaks',
- 'getStatements',
- ] )->getMock();
- $mockList->expects( $this->once() )->method( 'getByPropertyId'
)->willReturnSelf();
- $mockList->expects( $this->once() )->method( 'getMainSnaks'
)->willReturn( $snaks );
-
- $entity =
- $this->getMockBuilder(
StatementListProviderDummy::class )
- ->disableOriginalConstructor()
- ->getMock();
- $entity->expects( $this->once() )->method( 'getStatements'
)->willReturn( $mockList );
+ $entity = $this->getMockBuilder(
StatementListProviderDummy::class )
+ ->disableOriginalConstructor()
+ ->getMock();
+ $entity->expects( $this->once() )
+ ->method( 'getStatements' )
+ ->willReturn( $statementList );
$expected = [
'P123=STRING:testString',
--
To view, visit https://gerrit.wikimedia.org/r/382725
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib52a5fdf8e28cc17bb76757f0baa57b62aa14803
Gerrit-PatchSet: 6
Gerrit-Project: mediawiki/extensions/Wikibase
Gerrit-Branch: master
Gerrit-Owner: Thiemo Mättig (WMDE) <[email protected]>
Gerrit-Reviewer: Aude <[email protected]>
Gerrit-Reviewer: DCausse <[email protected]>
Gerrit-Reviewer: Daniel Kinzler <[email protected]>
Gerrit-Reviewer: Jonas Kress (WMDE) <[email protected]>
Gerrit-Reviewer: Ladsgroup <[email protected]>
Gerrit-Reviewer: Lucas Werkmeister (WMDE) <[email protected]>
Gerrit-Reviewer: Smalyshev <[email protected]>
Gerrit-Reviewer: Thiemo Mättig (WMDE) <[email protected]>
Gerrit-Reviewer: WMDE-leszek <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
