[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Arian_Bozorg]

Arian_Bozorg moved this task from Product Verification to Done on the Wikidata 
Dev Team (Wikidata.org Slice) board.
Arian_Bozorg removed Manuel as the assignee of this task.

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

WORKBOARD
  https://phabricator.wikimedia.org/project/board/6751/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Arian_Bozorg
Cc: Stevemunene, JAllemandou, Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, 
Aklapper, Michael, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, 
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Manuel]

Manuel closed this task as "Resolved".
Manuel claimed this task.
Manuel added a comment.


  Great, thank you all! I just opened a task to remove the clone: T351072: 
Remove the WDCM clone (stats1007) 

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Manuel
Cc: Stevemunene, JAllemandou, Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, 
Aklapper, Michael, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, 
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Michael]

Michael added a comment.


  Yes, it helps! We now no that there are no additional cronjobs running from 
this user (or related to WMDE in general). Which means that we should be able 
to safely remove that clone of the WDCM repo.

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Stevemunene, JAllemandou, Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, 
Aklapper, Michael, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, 
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Manuel]

Manuel added a comment.


  Thank you, @Stevemunene and @JAllemandou!
  
  @Michael: Did this help to come to a conclusion?

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Manuel
Cc: Stevemunene, JAllemandou, Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, 
Aklapper, Michael, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, 
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Stevemunene]

Stevemunene added a comment.


  In T350252#9307957 , 
@Manuel wrote:
  
  > Hi @JAllemandou, we would need to run `sudo -u analytics-wmde crontab -l` 
on stat1007 to verify that we can deprecate something that looks like a 
forgotten clone of some of the code that we last looked at together (for the 
Spark 3 migration). Could you please help us out and run the command for us, or 
refer us to someone that can?
  
  Thanks for reaching out @JAllemandou 
  Here's a list of the active wmde cron jobs.
  
stevemunene@stat1007:~$ systemctl list-timers |grep wmde
Thu 2023-11-09 08:31:00 UTC  23s leftThu 2023-11-09 08:30:01 
UTC  34s agowmde-analytics-minutely.timer   
wmde-analytics-minutely.service
Thu 2023-11-09 12:00:00 UTC  3h 29min left   Wed 2023-11-08 12:00:01 
UTC  20h agowmde-analytics-daily-noon.timer 
wmde-analytics-daily-noon.service
Fri 2023-11-10 03:00:00 UTC  18h leftThu 2023-11-09 03:00:01 
UTC  5h 30min ago   wmde-analytics-daily-early.timer
wmde-analytics-daily-early.service
Sun 2023-11-12 00:00:00 UTC  2 days left Sun 2023-11-05 00:00:01 
UTC  4 days ago wmde-analytics-weekly.timer 
wmde-analytics-weekly.service

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Stevemunene
Cc: Stevemunene, JAllemandou, Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, 
Aklapper, Michael, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, 
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Manuel]

Manuel added a subscriber: JAllemandou.
Manuel added a comment.


  Hi @JAllemandou, we would need to run `sudo -u analytics-wmde crontab -l` on 
stat1007 to verify that we can deprecate something that looks like a forgotten 
clone of some of the code that we last looked at together (for the Spark 3 
migration). Could you please help us out and run the command for us, or refer 
us to someone that can?

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Manuel
Cc: JAllemandou, Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, Aklapper, 
Michael, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, 
maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[AndrewTavis_WMDE]

AndrewTavis_WMDE added a comment.


  Just checked in with @Michael on this and we did a check of my ability to run 
`sudo -u analytics-wmde crontab -l` on `stat1007`. I sadly don't have access :(

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: AndrewTavis_WMDE
Cc: Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, 
ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Michael]

Michael moved this task from Ready for Peer Review to Product Verification on 
the Wikidata Dev Team (Wikidata.org Slice) board.
Michael added a comment.


  Thank you! This looks good to me. I'm moving it forward to Product 
Verification. @Manuel, @AndrewTavis_WMDE does one of you happen to have access 
to the `analytics-wmde` user so that you can check `sudo -u analytics-wmde 
crontab -l` on stat1007?

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

WORKBOARD
  https://phabricator.wikimedia.org/project/board/6751/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, 
ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE removed Lucas_Werkmeister_WMDE as the assignee of this 
task.
Lucas_Werkmeister_WMDE moved this task from In Development to Ready for Peer 
Review on the Wikidata Dev Team (Wikidata.org Slice) board.
Lucas_Werkmeister_WMDE added a comment.


  The clone path seems to resolve to `/srv/analytics-wmde/wdcm/src/`:
  
lucaswerkmeister-wmde@stat1007:~$ sudo -u analytics-wmde git -C 
/srv/analytics-wmde/wdcm/src/ remote -v
origin  https://gerrit.wikimedia.org/r/analytics/wmde/WDCM (fetch)
origin  https://gerrit.wikimedia.org/r/analytics/wmde/WDCM (push)
lucaswerkmeister-wmde@stat1007:~$ sudo -u analytics-wmde git -C 
/srv/analytics-wmde/wdcm/src/ show -U0
commit aa40182c322410584e91b93b9327adc017a218e9 (HEAD -> master, 
origin/master, origin/HEAD)
Author: GoranMilovanovic 
Date:   Thu Feb 8 20:08:09 2018 +0100

Minor

Change-Id: I5329a17b7b82b88958945f90b804195838e536b7

diff --git a/wdcmStructure_Update.R b/wdcmStructure_Update.R
index dcf4b79..1d2b102 100644
--- a/wdcmStructure_Update.R
+++ b/wdcmStructure_Update.R
@@ -291,0 +292,9 @@ write.csv(myWD$counts, "wdcmStructure_Counts.csv")
+### --- updateReport File
+updateReport <- as.character(Sys.time())
+upY <- substr(updateReport, 1, 4)
+upM <- as.numeric(substr(updateReport, 6, 7))
+upM <- month.name[upM]
+upD <- substr(updateReport, 9, 10)
+updateReport <- paste0(upY, " ", upM, " ", upD)
+write(updateReport, "updateReport.txt")
+
  
  Note that this commit is 23 commits behind the actually current master, if 
I’m not mistaken. But given that the repository thinks it’s up to date with 
`origin/master`, it seems the last `git fetch` was somewhere between February 
2018  (commit 
shown above) and June 2018 
 (first commit 
after that). I assume that’s because the Puppet spec has `ensure => 'present'` 
for the git clone, rather than `ensure => 'latest'` as seen in e.g. 
`graphite.pp` (which clones the analytics/wmde/scripts repo); the TODO below 
would agree with that:
  
git::clone { 'analytics/wmde/WDCM':
# TODO do we want a similar latest & production branch here? Or just 
manually pulling? scap?
# Currently when we update the code in the repo we will have to pull 
the updates ourselves.
ensure=> 'present',
  
  Given that one of the missing commits is called Security patch 
, I sure hope 
this outdated code never actually runs…
  
  None of the systemd timers seem to do anything with WDCM:
  
lucaswerkmeister-wmde@stat1007:~$ systemctl cat -- $(systemctl list-timers 
--no-legend --full | awk '{print $NF}') | grep -i wdcm | wc -l
0
  
  (`awk {print $NF}` selects the last field of the `systemctl list-timers` 
output, which is the name of the activated service, so this searches through 
all the timer-activated services.)
  
  Unfortunately, it looks like I can’t check `analytics-wmde`’s crontab:
  
lucaswerkmeister-wmde@stat1007:~$ sudo -u analytics-wmde crontab -l
crontabs/analytics-wmde/: fopen: Permission denied
  
  
  
  > Also, all the related cron-jobs that we know about 

 seem to run against a personal user directory, not something related to the 
`analytics-wmde` user?
  
  If we trust that this spreadsheet is complete, I think it’s almost certain 
that this repository isn’t used (now that we know what its path is, and can see 
that the path doesn’t show up in the spreadsheet); otherwise I guess we could 
ask someone with more permissions to check some more crontabs?
  
  (The other analytics clients 
 – stat1004, 
stat1005, stat1006, stat1008, stat1009 – don’t have a `/srv/analytics-wmde` 
directory at all, so I think this repository is limited to stat1007.)

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

WORKBOARD
  https://phabricator.wikimedia.org/project/board/6751/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, 
ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE claimed this task.
Lucas_Werkmeister_WMDE moved this task from Ready for Development to In 
Development on the Wikidata Dev Team (Wikidata.org Slice) board.
Lucas_Werkmeister_WMDE added a comment.


  I’ll see if I can make any sense of the puppet code.

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

WORKBOARD
  https://phabricator.wikimedia.org/project/board/6751/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Manuel, AndrewTavis_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, 
ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T350252: Figure out if the WDCM repository on stats1007 is actually used

[Michael]

Michael created this task.
Michael added projects: Wikidata Analytics, Wikidata, Wikidata Dev Team 
(Wikidata.org Slice).
Restricted Application added a subscriber: Aklapper.

TASK DESCRIPTION
  There is a puppet file to clone a Wikidata Concept Monitor repository to a 
stats100* host: wdcm.pp 

  
  It is unclear if that cloned repo is actually in use. It seems to come from 
an overall abandoned initiative: T171258: WDCM: Puppetization 

  
  Also, all the related cron-jobs that we know about 

 seem to run against a personal user directory, not something related to the 
`analytics-wmde` user?
  
  Then again, I do not yet understand how/where wdcm.pp 

 is invoked and with what arguments, particularly for `$dir`.
  
  **Acceptance criteria:**
  
  [ ] We know if the code cloned in wdcm.pp 

 is actually used or whether it can be deleted without any further effects at 
all.

TASK DETAIL
  https://phabricator.wikimedia.org/T350252

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Aklapper, Michael, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, 
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org