EBernhardson added a comment.
I suppose if we want to send all the properties to elasticsearch, but only have it index specific ones we can apply the keep words token filter to relationships.properties, i'm not seeing anything obvious for relationships itself. I thought pattern match might be able
Smalyshev added a comment.
@EBernhardson yes, this looks like what I've done in the patch, I just wondered if it's correct. Looks like it is then :)TASK DETAILhttps://phabricator.wikimedia.org/T175199EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
EBernhardson added a comment.
I think the analyzer was just pseudo code, to actually make it happen you need something like this: https://phabricator.wikimedia.org/P5975
That script outputs at the end
{
"relationships": [
"P1:Q1234",
"P31:Q54321",
"P31:Q7654"
],
Smalyshev added a comment.
@dcausse Could you explain a bit more how to set up the analyzer? I tried to figure how to do it but I'm not sure whether I did it right.TASK DETAILhttps://phabricator.wikimedia.org/T175199EMAIL
Pchelolo added a comment.
I wrote a little script to run through a sample of events in the job topics we have in prod right now and here's.a list of job types that had the releaseTimestamp set:
mediawiki.job.cdnPurge
mediawiki.job.cirrusSearchCheckerJob
mediawiki.job.cirrusSearchElasticaWrite
Pchelolo updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...```
There's another example that is **44 Mb is size** serialized. Kafka is capable of handling that, but it's not great in dealing with very large messages, so we can't increase the cap indefinitely. Maybe there's
gerritbot added a comment.
Change 376645 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/extensions/Wikibase@master] [WIP] Index statements on items
https://gerrit.wikimedia.org/r/376645TASK DETAILhttps://phabricator.wikimedia.org/T175199EMAIL
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T175199EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, gerritbotCc: gerritbot, debt, EBernhardson, dcausse, daniel, Aklapper, Smalyshev, Lordiis,
Pchelolo created this task.Pchelolo added projects: Services (doing), Wikidata.
TASK DESCRIPTIONAfter we've started posting the job events into #eventbus and Kafka, we've noticed that some of them were rejected by Kafka because of MESSAGE_SIZE_TOO_LARGE error. The limit was increased from 1 Mb to
Yurik updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...* press "Ctrl +" or "Ctrl -" (make fonts biggerer or smaller, ⌘+/⌘- on a mac)...TASK DETAILhttps://phabricator.wikimedia.org/T175312EMAIL
Yurik created this task.Yurik added a project: Wikidata-Query-Service.Herald added a subscriber: Aklapper.Herald added projects: Wikidata, Discovery.
TASK DESCRIPTIONRepo:
Open https://query.wikidata.org/
Click Examples
Observe that the word cloud is showing ok
Close Examples window
press Ctrl +
Lydia_Pintscher added a comment.
Let's wait a bit and see how things go on English Wiktionary and then go from there I'd say :)TASK DETAILhttps://phabricator.wikimedia.org/T175273EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc:
Framawiki added a comment.
Has there been a community consensus ? Is it good on this side ?TASK DETAILhttps://phabricator.wikimedia.org/T175273EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: FramawikiCc: Framawiki, Aklapper, Bugreporter, GoranSMilovanovic,
Ladsgroup added a comment.
It's still happening in wmf.17 too: https://logstash.wikimedia.org/goto/1f0b8c1062f060ec091c33b8a8e10e6b
I need to investigate this.TASK DETAILhttps://phabricator.wikimedia.org/T154555EMAIL
Hi Marco,
I guess this depends what you mean by "exhaustive". Exhaustive in that
every Wikidata item has ID X, or exhaustive in that we have every
instance of ID X in Wikidata?
The first is probably not going to happen, as the vast majority of
external identifiers have a defined scope for what
I guess this question for me is how do we do this in practice? How do we
make sure Wikidata stays up to date/synced with external databases we think
are important?
On 7 September 2017 at 20:51, Marco Fossati wrote:
> Hi everyone,
>
> As a data quality addict, I've been
matej_suchanek added a parent task: T109579: [Epic] Give more sister projects access to Wikidata.
TASK DETAILhttps://phabricator.wikimedia.org/T175273EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: matej_suchanekCc: Aklapper, Bugreporter, GoranSMilovanovic,
Hi everyone,
As a data quality addict, I've been investigating the coverage of
external identifiers linked to Wikidata items about people.
Given the numbers on SQID [1] and some SPARQL queries [2, 3], it seems
that even the second most used ID (VIAF) only covers *25%* of people
items circa.
Smalyshev added a project: User-Smalyshev.Smalyshev claimed this task.
TASK DETAILhttps://phabricator.wikimedia.org/T175199EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SmalyshevCc: debt, EBernhardson, dcausse, daniel, Aklapper, Smalyshev,
Ladsgroup added a comment.
This patch can go in when commons is on wmf.17. Sooner, it's useless. (See T174422: Make dbBatchSize in WikiPageUpdater configurable)TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
Jdlrobson moved this task from Needs Analysis to Tracking on the Readers-Web-Backlog board.Jdlrobson edited projects, added Readers-Web-Backlog (Tracking); removed Readers-Web-Backlog.
TASK
SandraF_WMF added a comment.
A rough first version is now ready and is being reviewed by several WMF colleagues.TASK DETAILhttps://phabricator.wikimedia.org/T173945EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: SandraF_WMFCc: Abit, BVershbow_WMF,
gerritbot added a comment.
Change 376562 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Reduce wikiPageUpdaterDbBatchSize to 20
https://gerrit.wikimedia.org/r/376562TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL
gerritbot added a comment.
Change 376558 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Introduce FormSet to simplify Lexeme state control
https://gerrit.wikimedia.org/r/376558TASK
Ladsgroup added a comment.
I made the batch smaller from 100 to 50 and I can do it to 20. Let me make a patch.TASK DETAILhttps://phabricator.wikimedia.org/T173710EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: LadsgroupCc: mobrovac, Nikerabbit, Mholloway,
debt triaged this task as "Normal" priority.debt edited projects, added Discovery-Search (Current work); removed Discovery-Search.
TASK DETAILhttps://phabricator.wikimedia.org/T173772EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: debtCc: Lydia_Pintscher,
Lydia_Pintscher closed this task as "Resolved".Lydia_Pintscher claimed this task.
TASK DETAILhttps://phabricator.wikimedia.org/T159316EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Stashbot, gerritbot, PokestarFan, TheDaveRoss, Vriullop,
Lydia_Pintscher closed subtask T159316: Enable arbitrary access on English Wiktionary as "Resolved".
TASK DETAILhttps://phabricator.wikimedia.org/T150178EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Bugreporter, Nizil, Esc3300, Aklapper,
debt edited projects, added Discovery-Search (Current work); removed Discovery-Search.
TASK DETAILhttps://phabricator.wikimedia.org/T173774EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Smalyshev, debtCc: gerritbot, Aklapper, daniel, Lydia_Pintscher,
debt moved this task from Needs triage to Up Next on the Discovery-Search board.debt triaged this task as "Normal" priority.debt added a comment.
This will help out with #structured-data-commons as well.TASK
gerritbot added a comment.
Change 376044 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] [WIP] Add form API module
https://gerrit.wikimedia.org/r/376044TASK
gerritbot added a comment.
Change 376540 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Add ChangeOp for Form addition
https://gerrit.wikimedia.org/r/376540TASK
gerritbot added a comment.
Change 376539 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Consider forms when compare Lexemes for equality
https://gerrit.wikimedia.org/r/376539TASK
Multichill added a comment.
This task doesn't read as a story, but more like recurring house keeping. Maybe rewrite it into something more actionable or create new more focused stories?TASK DETAILhttps://phabricator.wikimedia.org/T132690EMAIL
Lucas_Werkmeister_WMDE moved this task from Done to Review on the Wikidata-Sprint board.Lucas_Werkmeister_WMDE added a comment.
I’ve uploaded a second change to add a margin. Here’s what it looks like just before a line break:
F9376525: Screen Shot 2017-09-07 at 17.51.19.png
And after the line
Multichill closed this task as "Resolved".Multichill added a comment.
@Rfarrand I think this session and all other proposed sessions at https://phabricator.wikimedia.org/project/board/2530/ can be closed.TASK DETAILhttps://phabricator.wikimedia.org/T160828EMAIL
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T173742EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Aleksey_WMDE, gerritbotCc: gerritbot, Jonas, Aklapper, daniel, Lordiis, Cinemantique, GoranSMilovanovic, Adik2382,
gerritbot added a comment.
Change 376537 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Add addForm method to Lexeme
https://gerrit.wikimedia.org/r/376537TASK
gerritbot added a comment.
Change 376538 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseQualityConstraints@master] Add margin-left to gadget help link
https://gerrit.wikimedia.org/r/376538TASK
Aleksey_WMDE claimed this task.
TASK DETAILhttps://phabricator.wikimedia.org/T173742EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Aleksey_WMDECc: Jonas, Aklapper, daniel, Cinemantique, GoranSMilovanovic, QZanden, Izno, Wikidata-bugs, aude, Darkdadaah,
Reedy added a project: Wikimedia-Site-requests.
TASK DETAILhttps://phabricator.wikimedia.org/T159316EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ReedyCc: Stashbot, gerritbot, PokestarFan, TheDaveRoss, Vriullop, Nizil, Liuxinyu970226, Daniel_Carrero,
Reedy added a project: Wikimedia-Site-requests.
TASK DETAILhttps://phabricator.wikimedia.org/T175273EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ReedyCc: Aklapper, Bugreporter, GoranSMilovanovic, Jayprakash12345, QZanden, DatGuy, Devwaker, Urbanecm,
thiemowmde added a comment.
A break very close to the one shown in the image already happens. The CSS just needs an additional padding-left: 1em or something close to this. @Lucas_Werkmeister_WMDE, can you make this a patch? Otherwise please let me know if I should do it.TASK
Hanna_Petruschat_WMDE added a subscriber: Lydia_Pintscher.Hanna_Petruschat_WMDE added a comment.
Thanks for the very valid advice.
I have two other options now:
to help avoid text overlay, include another break if necessary. e.g. as follows
F9375487: 170907_text-link.png
establish a new
Lokal_Profil added a comment.
I believe the decision was to remove the DCAT-AP dumps from the dumping system as it is not well integrated and was not deemed critical enough to be worth the effort on doing so. @hoo?TASK DETAILhttps://phabricator.wikimedia.org/T163328EMAIL
mobrovac added a comment.
In T173710#3588015, @Joe wrote:
Wikibase refreshlinks jobs might benefit from being in smaller batches
+1 on this. As we have now all jobs being emitted to EventBus as well, we have had Kafka reject a portion of the jobs because they were larger than 4MB each. Upon
Hanna_Petruschat_WMDE added a comment.
Here is another attempt on how to handle the warnings:
the current icons measure something around 24*24 px
the solutions favored before would lose clarity in details
the screenshot attached shows two ways factoring in the original size:
the upper row shows
thiemowmde added a comment.
Thanks a lot for the detailed explanation! Makes sense, and helped me very much to understand the motivation better.
The main reason why I brought this up is: a text link needs localization, and the length of these localized texts might be very different in different
Bugreporter created this task.Bugreporter added a project: Wikidata.Herald added a subscriber: Aklapper.
TASK DESCRIPTIONSimilar to T159316: Enable arbitrary access on English WiktionaryTASK DETAILhttps://phabricator.wikimedia.org/T175273EMAIL
gerritbot added a comment.
Change 376515 had a related patch set uploaded (by Thiemo Mättig (WMDE); owner: Thiemo Mättig (WMDE)):
[mediawiki/extensions/Wikibase@master] Remove null special case from AliasesChangeOpDeserializer
https://gerrit.wikimedia.org/r/376515TASK
jcrespo added a comment.
Could, at least, that part have something to do with T164173, as a problem from the same cause, or a consequence of the fix? I also remember some tunning of some wikidata crons or job size, but not sure if upwards or downwards and not sure if related. Aaron, Ladsgroup or
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2017-09-07T13:13:06Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:376495|Add English Wiktionary as a client of Wikidata (T159316)]] (duration: 00m 49s)TASK
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2017-09-07T13:12:12Z] Synchronized dblists/wikidataclient.dblist: SWAT: [[gerrit:376495|Add English Wiktionary as a client of Wikidata (T159316)]] (duration: 00m 49s)TASK
gerritbot added a comment.
Change 376495 merged by jenkins-bot:
[operations/mediawiki-config@master] Add English Wiktionary as a client of Wikidata
https://gerrit.wikimedia.org/r/376495TASK DETAILhttps://phabricator.wikimedia.org/T159316EMAIL
Hanna_Petruschat_WMDE added a comment.
@Lucas_Werkmeister_WMDE : Thanks for bringing the discussion back to phabricator.
@thiemowmde mentioned on gerrit: "I think the label of the link should not be the text "help", but a question mark icon. Really, I mean that. What is the problem with a
Joe added a comment.
I did some more number crunching on the instances of runJob.php I'm running on terbium, I found what follows:
Wikibase refreshlinks jobs might benefit from being in smaller batches, as many of those are taking a long time to execute. Out of 33.4k wikibase jobs, we had the
Lucas_Werkmeister_WMDE added a comment.
@Krinkle thanks for your comments!
And this uses the same Sparql endpoint at https://query.wikidata.org/ as for public queries?
Yup.
but I didn't know it was used e.g. when saving edits (assuming validation happens there).
No, constraint checks are
Lucas_Werkmeister_WMDE moved this task from Review to Done on the Wikidata-Sprint board.Lucas_Werkmeister_WMDE added a subscriber: thiemowmde.Lucas_Werkmeister_WMDE added a comment.
Change merged, but @thiemowmde disagrees with the change… do you want to discuss this here?TASK
gerritbot added a comment.
Change 376495 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Add English Wiktionary as a client of Wikidata
https://gerrit.wikimedia.org/r/376495TASK DETAILhttps://phabricator.wikimedia.org/T159316EMAIL
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T159316EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, PokestarFan, TheDaveRoss, Vriullop, Nizil, Liuxinyu970226, Daniel_Carrero, jberkel,
Well you would have to drill down into the data to find their definitions
of bot users, but the conclusions seem to state that this is all pretty
premature anyway. If you look at the overall statistics that just measure
the basics (number of labels/descriptions/statements per item over time,
etc.)
WMDE-leszek added a comment.
Nice drawing @Addshore!TASK DETAILhttps://phabricator.wikimedia.org/T173225EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: thiemowmde, WMDE-leszekCc: Addshore, Jakob_WMDE, Lucas_Werkmeister_WMDE, hoo, aude, Ladsgroup, daniel,
WMDE-leszek added a comment.
Thanks for bringing this up @Ladsgroup. The thing that's been bugging me the most is that for more as a fairly new person it is not really clear which of all of those libs are something I should feel responsible for, and what. Having this information better included in
Ladsgroup added a comment.
And I did it in https://gerrit.wikimedia.org/r/376017 (but forgot some stuff I think)TASK DETAILhttps://phabricator.wikimedia.org/T174962EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: LadsgroupCc: Stashbot, gerritbot, Aklapper,
Addshore added a comment.
When writing subtasks of the kill the build ticket I drew the dependencies of all of our libs, see here attached:
F9370256: image.png
This might help us visualize what is going on :)TASK DETAILhttps://phabricator.wikimedia.org/T173225EMAIL
thiemowmde added a subscriber: WMDE-leszek.thiemowmde updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...Sum: ~3400 lines,500 lines
Not to forget these patches by @WMDE-leszek that remove local copies of jQuery and other libraries:
[]
thiemowmde updated the task description. (Show Details)
CHANGES TO TASK DESCRIPTION...[x] https://github.com/wmde/WikibaseDataModelJavaScript/pull/74 (−14 lines)...TASK DETAILhttps://phabricator.wikimedia.org/T172916EMAIL
thiemowmde added a comment.
I partly agree, but note that what you said is not covered by this ticket.
The most well maintained list of components is at http://wikiba.se/components/.
I believe that having lots of smaller libraries is not a problem in itself. The contrary. I believe it's a good
thiemowmde triaged this task as "Normal" priority.thiemowmde moved this task from incoming to ready to go on the Wikidata board.thiemowmde added projects: Need-volunteer, MediaWiki-extensions-WikibaseRepository.thiemowmde added subscribers: Lydia_Pintscher, Addshore.thiemowmde added a comment.
gerritbot added a comment.
Change 376256 merged by jenkins-bot:
[mediawiki/extensions/WikibaseQualityConstraints@master] Change help button in gadget to regular link
https://gerrit.wikimedia.org/r/376256TASK DETAILhttps://phabricator.wikimedia.org/T175153EMAIL
gerritbot added a comment.
Change 376200 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Let AliasesChangeOpDeserializer accept null in language aliases
https://gerrit.wikimedia.org/r/376200TASK DETAILhttps://phabricator.wikimedia.org/T175009EMAIL
dcausse added a comment.
deboosting can happen in the rescore stage, since we use a weighted sum we can either apply a negative penalty when relationship:P31:Q4167410 or a positive value when NOT relationship:P31:Q4167410.
Will we add all properties or just a set of selected properties?
thiemowmde added a comment.
This broke the $wgPropertySuggesterDeprecatedIds configuration for production.
This is sad. On https://gerrit.wikimedia.org/r/375999 I asked if this was taken care of, and got a "yes" by @Ladsgroup.TASK DETAILhttps://phabricator.wikimedia.org/T174962EMAIL
Jonas added a comment.
@Krinkle thanks for your input!
Maybe we should reopen T102752: [RFC] Workaround for checking the format constraintTASK DETAILhttps://phabricator.wikimedia.org/T173696EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: JonasCc: Krinkle,
The Springer paywall is no longer a problem for open science since there is
a certain Russian website, but in this case I see that we can find the full
article on ResearchGate:
Hoi,
Sorry but with only conclusions it is just that.. hidden behind a paywall.
Consequently it does not make a difference; our community cannot comment.
Please choose a different venue for publications.
Thanks,
GerardM
On 7 September 2017 at 08:37, Ettore RIZZA
Well, here is a fresh paper that seems to have been written to answer the
questions I had after this discussion.
" We performed a regression analysis to investigate how the contribution of
different types of users, i.e. bots and human editors, registered or
anonymous, influences outcome quality
Smalyshev added a comment.
I wonder also, is it possible to do the (de)boosting on rescore stage? The reason is because we can select different rescore profiles from URL (which means different widgets can use different boosts) while getting stuff added to the search query itself is more
78 matches
Mail list logo