[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-08-23 Thread Gehel
Gehel closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AKhatun_WMF, Gehel Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-27 Thread Maintenance_bot
Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AKhatun_WMF, Maintenance_bot Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Invadibot, MPhamWM

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-27 Thread dcausse
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AKhatun_WMF, dcausse Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Biggs657, Invadibot,

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-27 Thread gerritbot
gerritbot added a comment. Change 708282 **merged** by jenkins-bot: [wikimedia/discovery/analytics@master] bump rdf-spark-tools to 0.3.77 https://gerrit.wikimedia.org/r/708282 TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.o

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-27 Thread gerritbot
gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AKhatun_WMF, gerritbot Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Biggs657, Invadibot, Lalamarie6

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-27 Thread gerritbot
gerritbot added a comment. Change 708282 had a related patch set uploaded (by DCausse; author: DCausse): [wikimedia/discovery/analytics@master] bump rdf-spark-tools to 0.3.77 https://gerrit.wikimedia.org/r/708282 TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFEREN

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-26 Thread AKhatun_WMF
AKhatun_WMF claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AKhatun_WMF Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nan

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-26 Thread AKhatun_WMF
AKhatun_WMF added a comment. Joseph will suggest an optimization to this task when he is back. For now a simple `.distinct()` has been done on Spark dataframe to facilitate analysis on Wikidata dumps. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabri

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-26 Thread Maintenance_bot
Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Invadibot, MPhamWMF, maantieta

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-26 Thread gerritbot
gerritbot added a comment. Change 707051 **merged** by jenkins-bot: [wikidata/query/rdf@master] Deduplicate wikidata triples https://gerrit.wikimedia.org/r/707051 TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/p

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-22 Thread gerritbot
gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: gerritbot Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Biggs657, Invadibot, Lalamarie69, MPhamWMF,

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-22 Thread gerritbot
gerritbot added a comment. Change 707051 had a related patch set uploaded (by AKhatun; author: AKhatun): [wikidata/query/rdf@master] Deduplicate wikidata triples https://gerrit.wikimedia.org/r/707051 TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https:/

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-22 Thread CBogen
CBogen triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: CBogen Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Invadibot, MPhamWMF, maantietaja, CBogen, Akuck

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-12 Thread Maintenance_bot
Maintenance_bot added a project: Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: AKhatun_WMF, dcausse, Aklapper, JAllemandou, Invadibot, MPhamWMF, maantietaja, CBogen

[Wikidata-bugs] [Maniphest] T286436: Deduplicate triples when loading the wikibase RDF dumps into hive

2021-07-12 Thread dcausse
dcausse renamed this task from "Deduplicate tiples when loading wikibase RDF dataset into hive" to "Deduplicate triples when loading the wikibase RDF dumps into hive". TASK DETAIL https://phabricator.wikimedia.org/T286436 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/em