[Wikidata-bugs] [Maniphest] [Commented On] T238362: Blazegraph write performance tuning
Sj added a comment. Are these questions answered elsewhere? Seems to not quite fit the phab-task format TASK DETAIL https://phabricator.wikimedia.org/T238362 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Tarrow, dcausse, Igorkim78, Gehel, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T281854: Get baseline measurements/expectations for splitting scholarly articles from Wikidata
Sj added a comment. @Multichill the opening says "so that I can decide whether to move ahead with this plan and how to communicate it." -- it would help if that linked to a separate task, whose implementation depended on the outcome of this one. In the absence of that, this seems like the best and only? place in Phab to discuss the impacts of the split. @MPhamWMF Is this being evaluated as a one-off / one-time split, or is it a more general eval of the performance considerations from switching from a monolithic WD graph to a set of graph shards, with some max size (what's the rough range you imagine beyond which things stop scaling)? Any thoughts on performance implications of the latter may also be of interest to many of the large wikibase users, who regularly want to query a combination of at least one specialist base and WD itself, mediated by some query interface. TASK DETAIL https://phabricator.wikimedia.org/T281854 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Harej, Andrawaag, Lydia_Pintscher, Mohammed_Sadat_WMDE, nichtich, EgonWillighagen, Fnielsen, Darwinius, Daniel_Mietchen, Lokal_Profil, GoEThe, Alicia_Fagerving_WMSE, PKM, LWyatt, Multichill, Aklapper, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T281854: Get baseline measurements/expectations for splitting scholarly articles from Wikidata
Sj added a comment. @Multichill the opening says "so that I can decide whether to move ahead with this plan and how to communicate it." -- it would help if that linked to a separate task, whose implementation depended on the outcome of this one. In the absence of that, this seems like the best and only? place in Phab to discuss the impacts of the split. @MPhamWMF Is this being evaluated as a one-off / one-time split, or is it a more general eval of the performance considerations from switching from a monolithic WD graph to a set of graph shards, with some max size (what's the rough range you imagine beyond which things stop scaling)? Any thoughts on performance implications of the latter may also be of interest to many of the large wikibase users, who regularly want to query a combination of at least one specialist base and WD itself, mediated by some query interface. TASK DETAIL https://phabricator.wikimedia.org/T281854 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Harej, Andrawaag, Lydia_Pintscher, Mohammed_Sadat_WMDE, nichtich, EgonWillighagen, Fnielsen, Darwinius, Daniel_Mietchen, Lokal_Profil, GoEThe, Alicia_Fagerving_WMSE, PKM, LWyatt, Multichill, Aklapper, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T206561: Evaluate Virtuoso as alternative to Blazegraph
Sj added a comment. @Jerven: these are two great updates, thank you. Is there any more recent presentation you could share? Perhaps you could give a brown bag talk and record it as one :) Working with Kingsley + others @ OpenLink on evaluating this makes a lot of sense. TASK DETAIL https://phabricator.wikimedia.org/T206561 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Jerven, Base, TomT0m, Akuckartz, GreenReaper, Addshore, Lucas_Werkmeister_WMDE, Aklapper, Smalyshev, Invadibot, MPhamWMF, maantietaja, CBogen, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T201147: Expose constraint violations to WDQS using event queue
Sj added a comment. Looks promising and important -- usage and quality of constraints will certainly increase once this is resolved. TASK DETAIL https://phabricator.wikimedia.org/T201147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Lucas_Werkmeister_WMDE, Addshore, Smalyshev, Salgo60, MisterSynergy, Aklapper, Jonas, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Eihel, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, Esc3300, merbst, LawExplorer, _jensen, rosalieper, Agabi10, Scott_WUaS, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"
Sj added a comment. I get a 500 error now... will this be back up before the general release on Feb 1? TASK DETAIL https://phabricator.wikimedia.org/T297454 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcaro, Sj Cc: Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290300: Serve WCQS Sparql endpoint through api.wikimedia.org with OAuth 2
Sj added a comment. Agreed w/ Lucas, Husky, Multichill. And with Lego: "I would prefer a reduced SLO + no auth required over better uptime + auth" The existence of separate "higher SLO + higher red-tape service channel" makes sense -- we already have at least one. Why was that deemed unworkable in the past for WQDS? TASK DETAIL https://phabricator.wikimedia.org/T290300 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Multichill, LWyatt, Husky, Abbe98, FRomeo_WMF, GFontenelle_WMF, AntiCompositeNumber, Legoktm, Fuzheado, LucasWerkmeister, ttaylor, CBogen, Gehel, Aklapper, Zache, BPirkle, sdkim, hnowlan, RKemper, Zbyszko, Invadibot, MPhamWMF, maantietaja, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T285498: Wikidata: Investigate bibliographic bot
Sj added subscribers: JeanFred, Sj. Sj added a comment. Thanks for evaluating this. Can you suggest a path forward? Just making Aude.js work again? @JeanFred TASK DETAIL https://phabricator.wikimedia.org/T285498 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Daimona, Sj Cc: Sj, JeanFred, NRodriguez, Lydia_Pintscher, Samwilson, HMonroy, Aklapper, KSiebert, ldelench_wmf, Invadibot, maantietaja, Akuckartz, Nandana, Lahi, Gq86, dmaza, GoranSMilovanovic, QZanden, LawExplorer, JJMC89, B20180, _jensen, rosalieper, Scott_WUaS, Nakon, MusikAnimal, Wikidata-bugs, aude, Ricordisamoa, -jem-, Mbch331, Krenair ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"
Sj added a comment. Is this monitored by any of the status tools? Does it just need to be restarted? (and is it normal for projectadmin's to not have the ability to restart, or a feature of this being complex service?) TASK DETAIL https://phabricator.wikimedia.org/T297454 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, GFontenelle_WMF, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290839: Evaluate a double backend strategy for WDQS
Sj added a comment. In T290839#7360739 <https://phabricator.wikimedia.org/T290839#7360739>, @Justin0x2004 wrote: >> can give us the short update delays that users expect > > I am a user that rarely needs short update delays. > Didn't we just take a poll about what features of WDQS users prefer/want? Do we have the results of that to see if a double backend strategy would satisfy users? I am another user that doesn't need short update delays. Most of the interesting educational uses of Wikidata I can think of don't need those either. I would love to have a blazing fast SPARQL endpoint for a well indexed + cached snapshot. I would like but don't need to have a fast endpoint for the latest updates. If there were an option, I might use that for 10% of queries, trending on the simple side. Especially when it's understood that freshness has a speed cost. TASK DETAIL https://phabricator.wikimedia.org/T290839 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, driib, Jneubert, AndySeaborne, Addshore, Justin0x2004, Lucas_Werkmeister_WMDE, Bugreporter, Hannah_Bast, Aklapper, MPhamWMF, So9q, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T297995: Remove authentication from Wikimedia Commons Query Services (WCQS)
Sj added a comment. Looks like it is still 2-3 reqs/s. Is that the target usage rate? A fine metric for a query service is that it is used to its desired load. If underused, it could be made open-auth until this is reached, and then somehow have a conversation with those users re: options for rate limiting or uptime degradation. To the common need Husky + others note of wanting to build tools anyone can use w/o getting an error: for a given load you could split it between part for open usage and the rest for auth-only usage, on different clusters so the latter is available even when the former is brought down. [then tune the size of each cluster based on need, maybe one can go away completely if not used] TASK DETAIL https://phabricator.wikimedia.org/T297995 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Pigsonthewing, Ladsgroup, Elfix, Yug, pere_prlpz, Naseweis520, Stang, Denengelse, Spinster, Alicia_Fagerving_WMSE, RhinosF1, Andrawaag, MPhamWMF, Jane023, Sannita, Bugreporter, GFontenelle_WMF, Susannaanas, Gehel, AxelPettersson_WMSE, Abbe98, Izno, Pharos, Nikki, Ainali, Chicocvenancio, LWyatt, Legoktm, Husky, LucasWerkmeister, Fuzheado, Aklapper, Multichill, Danny_Benjafield_WMDE, Uata1122, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, maantietaja, Y.ssk, Muchiri124, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Ixocactus, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, El_Grafo, Dinoguy1000, Manybubbles, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T48556: [Epic] Wikidata 3rd party client (Instant Wikidata)
Sj added a comment. Definitely useful to communities I'm close to! Came across this recently in a wikicite / shared source metadata conversation. Instant WD, Instant Commons, and Q/P reuse in federation together will make strong reasons for people to choose MediaWiki when deciding on a platform for knowledge collab. TASK DETAIL https://phabricator.wikimedia.org/T48556 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Doc_James, Samwilson, Bawolff, MJL, Lens0021, jhsoby-WMNO, Mvolz, MGChecker, Asahiko, Aleksey_WMDE, Cparle, rajireturn, LJ, Gstupp, Daniel_Mietchen, RazShuty, Liuxinyu970226, Tgr, WMDE-leszek, ChristianKl, Tarawneh, aude, ArielGlenn, Wesalius, Agabi10, Tarrow, Addshore, Aklapper, Wikidata-bugs, Ricordisamoa, Lydia_Pintscher, me, Danny_Benjafield_WMDE, Astuthiodit_1, BeautifulBold, Suran38, karapayneWMDE, Invadibot, maantietaja, Peteosx1x, NavinRizwi, ItamarWMDE, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Dinoguy1000, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T324235: Give items in the Entity Schema namespace the same label editor as items and properties
Sj added a comment. @Lydia_Pintscher hurrah! TASK DETAIL https://phabricator.wikimedia.org/T324235 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Lydia_Pintscher, Michael, Bugreporter, So9q, Aklapper, RPI2026F1, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, SCIdude, Akuckartz, darthmon_wmde, pdehaye, Nandana, Lahi, Gq86, Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294133: Expose rdf-streaming-updater.mutation content through EventStreams
Sj added a subscriber: MPhamWMF. Sj added a comment. Thanks @Gehel for catching the duplicate. Suggested edits for this (merged) issue: - The merged issue should be for **making public the data WDQS uses for live updates**. A specific implementation such as "**...using EventStreams**" could be part of the discussion, or even a child task (trying that out as a way to make the data public). - @MPhamWMF set T330521 <https://phabricator.wikimedia.org/T330521> as high priority, and previously set this issue as high priority. If there is disagreement about that, it would be helpful to write a few words about that. Lowering the barrier to mirroring is critical for those maintaining live mirrors, and less so for those only working on WD and WDQS core. But committing to make this work could open avenues to share WDQS loads which would affect core performance and disaster planning (the context in which this came up recently). TASK DETAIL https://phabricator.wikimedia.org/T294133 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: MPhamWMF, KingsleyIdehen, Gehel, Jheald, tfmorris, danshick-wmde, Sj, RBrounley_WMF, Harej, Ottomata, Aklapper, dcausse, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, EChetty, lbowmaker, BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, Chlod, CBogen, ItamarWMDE, Akuckartz, 4748kitoko, holger.knust, Nandana, Namenlos314, Akovalyov, Lahi, Gq86, Xinbenlv, Vacio, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Nirmos, JAllemandou, terrrydactyl, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331, Krenair, jeremyb ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330525: Migrate Wikidata off of Blazegraph
Sj updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T330525 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Gehel, Aklapper, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330521: Make WDQS update stream public
Sj updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T330521 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Aklapper, Sj, AWesterinen, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330521: Make WDQS update stream public
Sj added subscribers: KingsleyIdehen, Gehel, Jheald. Sj added a comment. cc @Gehel @KingsleyIdehen @Jheald TASK DETAIL https://phabricator.wikimedia.org/T330521 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Jheald, Gehel, KingsleyIdehen, Aklapper, Sj, AWesterinen, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330525: Migrate off of Blazegraph
Sj created this task. Sj added projects: Wikidata-Query-Service, Wikidata Analytics, Wikidata. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION Currently there are a lot of issues for evaluation and analysis, such as: T206560 <https://phabricator.wikimedia.org/T206560> - Evaluate alternatives to BG (including lots of subtasks around testing and evaluating alternatives) T306725 <https://phabricator.wikimedia.org/T306725> - Decide which BG services to migrate(assuming a migration is bound to happen) but nothing for the migration itself, which cannot be avoided. Hence this task. [Meanwhile there are more issues opening every season for fixing slowness, failures, or other inconsistencies with Blazegraph, an ongoing tax for not migrating.] The time is ripe: Blazegraph instability is slowing down data reloads on WDQS, and may prevent them altogether in the future: @Gehel wrote <https://lists.wikimedia.org/hyperkitty/list/wikid...@lists.wikimedia.org/message/7QTJBRU2T3J22SNV4TGBRML4QNBGCEOU/>: > TL;DR: We expect to successfully complete the recent data reload on Wikidata Query Service soon, but we've encountered multiple failures related to the size of the graph, and anticipate that this issue may worsen in the future. Although we succeeded this time, we cannot guarantee that future reload attempts will be successful given the current trend of the data reload process. Thank you for your understanding As the Query Service is the public-facing part of Wikidata in many contexts, this feels like preventing WD itself from being updated. Proposal: implement a production-quality migration before we next need to reload the query service. Even if there is a double-backend solution for a time (T290839 <https://phabricator.wikimedia.org/T290839>), or the initial migration exposes a need to change the migration strategy or change the target backend, having even a temporary alternative may provide an alternative way to resolve queries. And working through the migration process would bring needed attention to this critical step in WD growth, and may help practice future recovery workflows. TASK DETAIL https://phabricator.wikimedia.org/T330525 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Gehel, Aklapper, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330525: Migrate Wikidata off of Blazegraph
Sj renamed this task from "Migrate off of Blazegraph" to "Migrate Wikidata off of Blazegraph". Sj updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T330525 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Gehel, Aklapper, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330521: Make WDQS update stream public
Sj created this task. Sj added projects: Wikidata-Query-Service, Wikidata Analytics. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION **Feature summary** (what you would like to be able to do and where): Currently, the WDQS update process is mostly decoupled from the backend. It produces a stream of RDF updates that is backend independent, and has a thin Blazegraph-specific adapter to load the data into Blazegraph. From a description of Wikidata dumps and resources, it would be useful to have a link to such an update stream, updated in near-real time. Guillaume Lederrey wrote: "//That data stream isn't exposed publicly. There are a few tricky part about the stream needing to be synchronized with a specific Wikidata dump that makes it not entirely trivial to reuse... But if there is enough interest, we could potentially work on making that stream public.//" **Use case(s)** (list the steps that you performed to discover that problem, and describe the actual underlying problem which you want to solve. Do not describe only a solution): This would enable 3rd-party WDQS mirrors to maintain near-real-time updates of their own query services, on a variety of backends. **Benefits** (why should this be implemented?): This would enable a community federation of data mirrors, which could support a higher query load, and be robust to downtime of any of its members. It would also pave the way to discussing a future FR making that federation more visible to queriers: From a WDQS UI, an advanced option could let one choose one's data backend, or for the interface to suggest appropriate backends if your query can't be resolved on / uses non-standard features not available on / keeps timing out on the currently chosen backend]. TASK DETAIL https://phabricator.wikimedia.org/T330521 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Aklapper, Sj, AWesterinen, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330521: Make WDQS update stream public
Sj updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T330521 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Jheald, Gehel, KingsleyIdehen, Aklapper, Sj, AWesterinen, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T244847: Future of the OpenRefine Wikidata reconciliation interface
Sj added a comment. @Pintoch The ontotext instance is interesting. Might they be interested in also providing something closer to a WQDS interface? TASK DETAIL https://phabricator.wikimedia.org/T244847 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, VladimirAlexiev, Tarrow, Michael, GFontenelle_WMF, jdtoy, SandraF_WMF, JeanFred, Fuzheado, Eugene233, PabloCastellano, Loz.ross, RShigapov, Spinster, Samantha_Alipio_WMDE, WMDE-leszek, Addshore, Lydia_Pintscher, Abbe98, tfmorris, David_Haskiya_WMSE, Lokal_Profil, Mvolz, Alicia_Fagerving_WMSE, Regisrob, Pintoch, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Thadguidry, Tore_Danielsson_WMSE, Nandana, Lahi, Gq86, GoranSMilovanovic, Nattes, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Nikerabbit, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T330525: Migrate Wikidata off of Blazegraph
Sj added a comment. Hopefully this is already covered somewhere else and can be closed and merged! I'd hope for the equivalent to include: - A rough timeline for migration - Current status of the decisions made / pending re: where and how to migrate - A map of other components + services affected, so they can be notified and run their own downstream analysis Other useful subparts, which may depend on the details above: - Plans for handling possible obstacles or failures along the way - Metrics run on test datasets, and then on the full system post-migration, to see how it's working - Current challenges and stopgaps that should no longer be needed (technical + social debt) - Current stalled feature or service requests that should become possible TASK DETAIL https://phabricator.wikimedia.org/T330525 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Sj, Gehel, Aklapper, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T244847: Future of the OpenRefine Wikidata reconciliation interface
Sj added a comment. @Addshore thank you for writing that and resurfacing your old post. Agreed that recon should be core to Wikibase, both for it to reach new users and to make setting up new small projects easier. TASK DETAIL https://phabricator.wikimedia.org/T244847 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Nikerabbit, EBernhardson, Sj, VladimirAlexiev, Tarrow, Michael, GFontenelle_WMF, jdtoy, SandraF_WMF, JeanFred, Fuzheado, Eugene233, PabloCastellano, Loz.ross, RShigapov, Spinster, Samantha_Alipio_WMDE, WMDE-leszek, Addshore, Lydia_Pintscher, Abbe98, tfmorris, David_Haskiya_WMSE, Lokal_Profil, Mvolz, Alicia_Fagerving_WMSE, Regisrob, Pintoch, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Thadguidry, Tore_Danielsson_WMSE, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Nattes, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T48556: [Epic] Wikidata 3rd party client (Instant Wikidata)
Sj added a comment. @michael forgive my confusion, isn't part of this being able to use one Repository with multiple Clients, or to enable WikibaseClient to draw from both a local Repository and the global Wikidata repository? TASK DETAIL https://phabricator.wikimedia.org/T48556 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: Michael, Abbe98, Sj, Doc_James, Samwilson, Bawolff, MJL, Lens0021, jhsoby-WMNO, Mvolz, MGChecker, Asahiko, Aleksey_WMDE, Cparle, rajireturn, LJ, Gstupp, Daniel_Mietchen, RazShuty, Liuxinyu970226, Tgr, WMDE-leszek, ChristianKl, Tarawneh, aude, ArielGlenn, Wesalius, Agabi10, Tarrow, Addshore, Aklapper, Wikidata-bugs, Ricordisamoa, Lydia_Pintscher, me, Danny_Benjafield_WMDE, Astuthiodit_1, BeautifulBold, Suran38, karapayneWMDE, Invadibot, maantietaja, Peteosx1x, NavinRizwi, ItamarWMDE, Akuckartz, Nandana, Harideepan, Lahi, Gq86, GoranSMilovanovic, alaa, QZanden, KimKelting, LawExplorer, _jensen, rosalieper, Esh77, CFCF, Scott_WUaS, FloNight, Dinoguy1000, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T356773: [tracking] Community feedback for the WDQS Split the Graph project
Sj added a comment. One reason is that citations are a large corpus with a fairly narrow range of schemas and uses, so it could conceivably be implemented with optimizations that can't be applied across the board. TASK DETAIL https://phabricator.wikimedia.org/T356773 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sannita, Sj Cc: Physikerwelt, EgonWillighagen, ArthurPSmith, Sj, dcausse, valerio.bozzolan, tfmorris, Gehel, Aklapper, Danny_Benjafield_WMDE, S8321414, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, KimKelting, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T297995: Remove authentication from Wikimedia Commons Query Services (WCQS)
Sj added a comment. Has there been any recent progress on this? T348269 <https://phabricator.wikimedia.org/T348269> by one of the preeminent reusers of SDC, in both scale and impact, seems relevant; but that remains untriaged and this one is High but unassigned. Usage seems surprisingly steady at 2.5-3 q/s, is there any information about how it's currently being used? cc @ladsgroup @Spinster @Fuzheado TASK DETAIL https://phabricator.wikimedia.org/T297995 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Sj Cc: RP88, Sj, Pigsonthewing, Ladsgroup, Elfix, Yug, pere_prlpz, Naseweis520, Stang, Denengelse, Spinster, Alicia_Fagerving_WMSE, RhinosF1, Andrawaag, Jane023, Sannita, Bugreporter, GFontenelle_WMF, Susannaanas, Gehel, AxelPettersson_WMSE, Abbe98, Izno, Pharos, Nikki, Ainali, Chicocvenancio, LWyatt, Legoktm, Husky, LucasWerkmeister, Fuzheado, Aklapper, Multichill, Danny_Benjafield_WMDE, Uata1122, S8321414, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, maantietaja, Y.ssk, FRomeo_WMF, Muchiri124, ItamarWMDE, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, KimKelting, merbst, LawExplorer, _jensen, rosalieper, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Ixocactus, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Daniel_Mietchen, El_Grafo, Dinoguy1000, Manybubbles, Lydia_Pintscher, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
