| Addshore renamed this task from "Migrate search integration of article place holder to elastic" to "Migrate search integration of ArticlePlaceholder to elastic". Addshore added a subscriber: Legoktm. Addshore added a project: wikidata-tech-focus. Addshore updated the task description. (Show Details) |
CHANGES TO TASK DESCRIPTION
As * I want to kill the wb_terms table so that it will stop exploding
**Details**
ArticlePlaceholder currently uses the wb_terms table for search.
The use of wb_terms has caused us many issues over the years, causing an outage[[ https://wikitech.wikimedia.org/wiki/Incident_documentation/20180524-wikidata | rather recently (May 2018) ]].
As a result the ArticlePlaceholder search integration is currently disabled (it will be re enabled with T195751).
In order to re enable the feature and ensure stability and reliability we should first migrate the feature away from wb_terms and instead use elasticsearch.
Using elasticsearch directly in PHP poses some difficulties as all of the elastic & search code is in Repo and ArticlePlacholder only has access to Client.
During a discussion with @Legoktm, @Aleksey_WMDE & I Wikimania this year we discussed the idea of calling the repo wbsearchentities API internally from the client, this is probably the best path forward.
**Impact & Priority**
The feature should probably not be enabled again until this task is resolved, hence T195751 is blocked by this.
As for the priority of T195751 (turning the feature back on) that is down to @Lydia_Pintscher.
When writing this description this task was marked as "High" so I'll leave it as that for now (2018-08-19)
**Task**
Make ArticlePlaceholder use elasticsearch instead of the wb_terms table by calling the repo wbsearchentities API module.
**Acceptance criteria**
- ArticlePlaceholder on longer uses the wb_terms table for anything
- ArticlePlaceholder uses wikidata search via the wbsearchentities API module
- Calls to the API module should have a short timeout and should fail gracefully & log success / failure rates
**Details**
ArticlePlaceholder currently uses the wb_terms table for search.
The use of wb_terms has caused us many issues over the years, causing an outage[[ https://wikitech.wikimedia.org/wiki/Incident_documentation/20180524-wikidata | rather recently (May 2018) ]].
As a result the ArticlePlaceholder search integration is currently disabled (it will be re enabled with T195751).
In order to re enable the feature and ensure stability and reliability we should first migrate the feature away from wb_terms and instead use elasticsearch.
Using elasticsearch directly in PHP poses some difficulties as all of the elastic & search code is in Repo and ArticlePlacholder only has access to Client.
During a discussion with @Legoktm, @Aleksey_WMDE & I Wikimania this year we discussed the idea of calling the repo wbsearchentities API internally from the client, this is probably the best path forward.
**Impact & Priority**
The feature should probably not be enabled again until this task is resolved, hence T195751 is blocked by this.
As for the priority of T195751 (turning the feature back on) that is down to @Lydia_Pintscher.
When writing this description this task was marked as "High" so I'll leave it as that for now (2018-08-19)
**Task**
Make ArticlePlaceholder use elasticsearch instead of the wb_terms table by calling the repo wbsearchentities API module.
**Acceptance criteria**
- ArticlePlaceholder on longer uses the wb_terms table for anything
- ArticlePlaceholder uses wikidata search via the wbsearchentities API module
- Calls to the API module should have a short timeout and should fail gracefully & log success / failure rates
TASK DETAIL
EMAIL PREFERENCES
To: Addshore
Cc: Legoktm, Aleksey_WMDE, Aklapper, Addshore, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, GoranSMilovanovic, lisong, QZanden, LawExplorer, Salgo60, Jonas, Wikidata-bugs, aude, Mbch331
Cc: Legoktm, Aleksey_WMDE, Aklapper, Addshore, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, GoranSMilovanovic, lisong, QZanden, LawExplorer, Salgo60, Jonas, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
