[Wikidata-bugs] [Maniphest] T358311: Check home/HDFS leftovers of goransm (timeboxed 0,5 days)

Tue, 21 May 2024 03:27:50 -0700 

AndrewTavis_WMDE added a comment.


  ⚠️ Currently WIP ⚠️
  ===================
  
  Going through the files sent by @JAllemandou above 
<https://phabricator.wikimedia.org/T358311#9648470>. This message will be saved 
as I go so that I don't loose my progress 😊 If I do find something worth 
documenting, then I'll also include it below so that this task can serve as a 
reference for later if need be.
  
  stat1004
  --------
  
  All of the files are not worth keeping. See descriptions and reasoning below:
  
    total 28
    
    Analytics
    └─ NewEditors 
        └─ adHoc (nothing of interest)
        └─ Compaigns
            └─ 2019 and 2020 email compaigns with R based analysis (nothing of 
interest)
    └─ WDCM
        └─ WDCM_Output 
            └─ Lots directories of CSVs (nothing of interest)
        └─ WDCM_Scripts
            └─ R based scripts that would be archived on Gerrit if they were 
ever in production (nothing of interest)
    └─ Wikidata
        └─ misc
            └─ Some ad hoc work (nothing of interest)
        └─ WD_languagesLandscape
            └─ R based scripts that would be archived on Gerrit if they were 
ever in production (nothing of interest)
        └─ WD_ORES_ItemQuality (nothing of interest given Lift Wing migration)
        └─ WD_UsageCoverage
            └─ R and Python scripts that are doubtless versions of the WDCM 
UsageCoverage dashboard that's archived on Gerrit (nothing of interest)
    Experiments
        └─ Empty
    _miscWMDE
        └─ summerBannerCampaign2017_DataOUT
            └─ TSV files (nothing of interest)
        └─ TWLBanner_2017
            └─ TSV files and simple HQL queries from `wmf.webrequest` for 
banner campaigns hits (nothing of interest, easy to learn as needed)
    
    Example query:
    
    SELECT count(*)
    FROM wmf.webrequest
    WHERE uri_host = 'de.wikipedia.org'
      AND uri_query LIKE "$/wiki/Wikipedia:Umfragen/Technische_WΓΌnsche_2017$"
      AND http_method = 'GET'
      AND is_pageview = TRUE
      AND YEAR = 2017
      AND MONTH = 6
      AND DAY = 1
      and HOUR = 20;
    
        └─ TWLBanner_2017_DataOUT
            └─ TSV files (nothing of interest)
    _miscWMDE_1004
        └─ TWLBanner_2017
            └─ One HQL and one TSV file that are similar to the above (nothing 
of interest)
    R
        └─ x86_64-pc-linux-gnu-library (nothing of interest)
    Research
        └─ DydimusZengenene
            └─ Note: work to support a researcher (nothing of interest)
            └─ _analytics
            └─ _data
            └─ DydimusZengenene.Rproj
            └─ ParseTargetPage.R
    wdUsagePerPage
        └─ Related to the percentage usage dashboard, so would be archived on 
Gerrit if they were ever in production (nothing of interest)
  
  
  
  stat1005
  --------
  
    total 964
    
    Analytics
        └─ 
    BotEdits_perProject.ipynb
        └─ 
    crontabstat1005.txt
        └─ 
    DataModelTerms_20210228_Updates.ipynb
        └─ 
    dewiki_NewEds_2021.ipynb
        └─ 
    QCF_M2_Test.ipynb
        └─ 
    QuratorCuriousFacts_Separators.ipynb
        └─ 
    Qurator_M1.ipynb
        └─ 
    R
        └─ 
    snapshot_query.hql
        └─ 
    Untitled1.ipynb
        └─ 
    untitled1.txt
        └─ 
    Untitled2.ipynb
        └─ 
    Untitled3.ipynb
        └─ 
    Untitled4.ipynb
        └─ 
    Untitled5.ipynb
        └─ 
    Untitled.ipynb
        └─ 
    untitled.txt
        └─ 
    venv
        └─ 
    wd_cluster_fetch_items_M2.ipynb
        └─ 
    wd_cluster_fetch_items_M3.ipynb
        └─ 
    WDCM_ETL_OTHER_TEST.ipynb
        └─ 
    WDCM_Statements_Test.ipynb
        └─ 
    WD_HumanEditsPerClass_RevisionTags.ipynb
        └─ 
    WD_Inequality_Intake.ipynb
        └─ 
    WD_Languages_Datamodel_CollectInit.ipynb
        └─ 
    WD_Languages_Datamodel_EXP.ipynb
        └─ 
    WD_MonthlyEditors.ipynb
        └─ 
    WD_Sitelinks_WDAHP_202108.ipynb
        └─ 
    wd_statements_HiveQL_Query.hql
        └─ 
    WD_Translations.ipynb
        └─ 
    WHEIP_exps.ipynb
        └─ 
    wikidata_analytics_examples
        └─ 
    WikidataRevisions_November2020.csv
        └─ 
  
  
  
  stat1006
  --------
  
    total 48
    
    misc_projects
        └─ 
    myTemp
        └─ 
    NewEds
        └─ 
    nohup.out
        └─ 
    R
        └─ 
    RPckg
        └─ 
    RScripts
        └─ 
    sqlIn
        └─ 
    sqlOut
        └─ 
    WDCM_Credentials
        └─ 
    WDCM_DataIN
        └─ 
    WDCM_DataOUT
        └─ 
    WDCM_sql
        └─ 
  
  
  
  stat1007
  --------
  
    total 28
    
    Analytics
        └─ 
    crontabstat1007.txt
        └─ 
    Experiments
        └─ 
    Python3
        └─ 
    R
        └─ 
    RScripts
        └─ 
    venv
        └─ 
  
  
  
  stat1008
  --------
  
    total 16
    
    Analytics
        └─ 
    R
        └─ 
    renv
        └─ 
    venv
        └─ 
  
  
  
  stat1009
  --------
  
    total 0
  
  
  
  stat1010
  --------
  
    total 0
  
  
  
  HDFS
  ----
  
    Found 55 items
    
    /user/goransm/.Trash
        └─ 
    /user/goransm/.metadata
        └─ 
    /user/goransm/.sparkStaging
        └─ 
    /user/goransm/.staging
        └─ 
    /user/goransm/.temp
        └─ 
    /user/goransm/Architectural-Structure_ItemIDs.csv
        └─ 
    /user/goransm/Astronomical-Object_ItemIDs.csv
        └─ 
    /user/goransm/Book_ItemIDs.csv
        └─ 
    /user/goransm/Chemical-Entities_ItemIDs.csv
        └─ 
    /user/goransm/Event_ItemIDs.csv
        └─ 
    /user/goransm/Gene_ItemIDs.csv
        └─ 
    /user/goransm/Geographical-Object_ItemIDs.csv
        └─ 
    /user/goransm/Human_ItemIDs.csv
        └─ 
    /user/goransm/ORESPredictions
        └─ 
    /user/goransm/Organization_ItemIDs.csv
        └─ 
    /user/goransm/Taxon_ItemIDs.csv
        └─ 
    /user/goransm/Thoroughfare_ItemIDs.csv
        └─ 
    /user/goransm/WDCM_Biases_ETL_Test
        └─ 
    /user/goransm/WDCM_CollectedGeoItems
        └─ 
    /user/goransm/WDCM_CollectedItems
        └─ 
    /user/goransm/Wikimedia_Internal_ItemIDs.csv
        └─ 
    /user/goransm/Work-Of-Art_ItemIDs.csv
        └─ 
    /user/goransm/dewiki_revisions
        └─ 
    /user/goransm/dfTrain1.csv
        └─ 
    /user/goransm/dfTrain2.csv
        └─ 
    /user/goransm/dfTrain3.csv
        └─ 
    /user/goransm/dfTrain4.csv
        └─ 
    /user/goransm/dfTrain5.csv
        └─ 
    /user/goransm/flights.csv
        └─ 
    /user/goransm/mysql-analytics-research-client-pw.txt
        └─ 
    /user/goransm/refClassSubclasses.csv
        └─ 
    /user/goransm/separators.csv
        └─ 
    /user/goransm/singleValueConstraintProperties.csv
    /user/goransm/subclasses.csv
        └─ 
    /user/goransm/tfMatrixDF.csv
        └─ 
    /user/goransm/tfMatrix_Human.csv
        └─ 
    /user/goransm/wdORESQuality.csv
        └─ 
    /user/goransm/wdORESQuality_Reuse.csv
        └─ 
    /user/goransm/wdORESQuality_Reuse_Commons.csv
        └─ 
    /user/goransm/wdORESQuality_Reuse_nonCommons.csv
        └─ 
    /user/goransm/wd_dump_geocoded
        └─ 
    /user/goransm/wd_dump_human_author
        └─ 
    /user/goransm/wd_dump_human_creator
        └─ 
    /user/goransm/wd_dump_human_gender
        └─ 
    /user/goransm/wd_dump_human_occupation
        └─ 
    /user/goransm/wd_dump_item_language
        └─ 
    /user/goransm/wd_dump_labels_English
        └─ 
    /user/goransm/wd_entity_reuse
        └─ 
    /user/goransm/wd_extId_data_qual_.csv
        └─ 
    /user/goransm/wd_extId_data_ref_.csv
        └─ 
    /user/goransm/wd_extId_data_ref_snak_.csv
        └─ 
    /user/goransm/wd_extId_data_stat_.csv
        └─ 
    /user/goransm/wdcmsqoop
        └─ 
    /user/goransm/wdtranslationsb
        └─ 
    /user/goransm/wikidataRevisions_EXP.csv
        └─ 
  
  
  
  Hive
  ----
  
  Nothing found

TASK DETAIL
  https://phabricator.wikimedia.org/T358311

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: AndrewTavis_WMDE
Cc: brouberol, JAllemandou, MoritzMuehlenhoff, Manuel, Aklapper, 
AndrewTavis_WMDE, Danny_Benjafield_WMDE, S8321414, Astuthiodit_1, BTullis, 
karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Dringsim, 
Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, 
_jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331

_______________________________________________
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

Reply via email to