[Cloud] Parsoid APIs?

I know there's been a ton of work done of Parsoid lately. This is great, and the amount of effort that's gone into this functionality is really appreciated. It's clear that Parsoid is the way of the future, but the documentation of how you get a Parsoid parse tree via an AP call isI kind of co

Re: [Cloud] Logging vis-a-vis WMF privacy policy?

Yup, I'm using django. I was not previously aware of django-log-request-id, but that looks like exactly what I had in mind, so probably what I'll use. Thanks for the pointer. > On Jul 28, 2020, at 2:51 PM, Bryan Davis wrote: > > You did > not mention a specific language that you are using, b

[Cloud] Logging vis-a-vis WMF privacy policy?

Is there any specific guidance for what's appropriate or not appropriate for my tool to be logging? I'm using OAUTH, so my tool knows the identity of the user. I assume I don't want to be logging that. Or, is it OK, as long as the logs stay within toolforge? For debugging purposes, I'd like

[Cloud] What to do with obsolete oauth consumer keys?

I've created a handful of keys, experimenting the various configs. Now that I've figured out what I need, I have a bunch of old consumers that I don't need any more. Is there anything I should do to revoke them? ___ Wikimedia Cloud Services mailing

Re: [Cloud] Where to look up my oauth tokens?

deleted? Is there such a thing as a "read-only" consumer? For what I want to do, I'll never want my app to do any edits. It would make sense to get a read-only consumer key, if such a thing existed. > On Jul 9, 2020, at 7:29 PM, Roy Smith wrote: > > I've got

Re: [Cloud] Where to look up my oauth tokens?

Yup, that's exactly what I was looking for. Thanks. > On Jul 9, 2020, at 8:56 PM, Bryan Davis wrote: > > On Thu, Jul 9, 2020 at 5:29 PM Roy Smith wrote: >> >> I've got several oauth tokens that I registered for a while ago. I have the >> actual

[Cloud] Where to look up my oauth tokens?

I've got several oauth tokens that I registered for a while ago. I have the actual tokens, but where I can I look up the details like whether I registered them as owner-only, etc? ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (f

Re: [Cloud] Toolforge: new domain toolforge.org

do with allowed hosts? What are allowed hosts and where are > they configured? > > Thanks, > -Travis > > On Tue, Jul 7, 2020 at 7:02 AM Martin Urbanec <mailto:martin.urba...@wikimedia.cz>> wrote: > Yes, you need to add the host to the allowed hosts. > > Mar

Re: [Cloud] Toolforge: new domain toolforge.org

I've started getting: > Invalid HTTP_HOST header: 'spi-tools-dev.toolforge.org'. You may need to add > 'spi-tools-dev.toolforge.org' to ALLOWED_HOSTS. I assume that's related to the change? > On Jul 7, 2020, at 6:50 AM, Arturo Borrero Gonzalez > wrote: > > On 2020-07-06 17:59, Arturo Borre

Re: [Cloud] Slow uWSGI logging

https://phabricator.wikimedia.org/T256426 <https://phabricator.wikimedia.org/T256426> > On Jun 25, 2020, at 4:00 PM, YiFei Zhu wrote: > >> Should I open a phab ticket on this? > > Yes please. > > On Thu, Jun 25, 2020 at 2:55 PM Roy Smith wrote: >> >

Re: [Cloud] Slow uWSGI logging

g the data... > > YiFei Zhu > > On Thu, Jun 25, 2020 at 8:12 AM Roy Smith wrote: >> >> https://tools.wmflabs.org/spi-tools/spi/ >> >> To generate a request, click the "Sock Info" button or the "Index" link. >> >> The log file is

Re: [Cloud] Slow uWSGI logging

tool is this? I can probably strace and see if that is the case. > > YiFei Zhu > > > On Wed, Jun 24, 2020 at 10:16 PM Roy Smith wrote: >> >> I'm running a web server with "webservice --backend=kubernetes python3.7". >> As I tail the uwsgi.log fi

Re: [Cloud] Slow uWSGI logging

rote: > > My guess is output stream buffering, at either the libc level or the > python level. > > Which tool is this? I can probably strace and see if that is the case. > > YiFei Zhu > > > On Wed, Jun 24, 2020 at 10:16 PM Roy Smith wrote: >> >> I

[Cloud] Slow uWSGI logging

I'm running a web server with "webservice --backend=kubernetes python3.7". As I tail the uwsgi.log file, requests to my server get logged with very long delays. I just timed one at about a minute and a half between when the request was served (03:07:57 UTC 2020) and when it showed up in the lo

Re: [Cloud] Need help with selectize.js

think there's like some basic software engineering lesson in there :-) > On Jun 22, 2020, at 10:01 PM, Roy Smith wrote: > > I'm mostly a python guy, who only dabbles in javascript when I can't avoid it > :-) > > I've got a django-based tool (https://tools.wm

Re: [Cloud] Need help with selectize.js

#x27;m assuming) that will provide some protection against the file being unexpectedly replaced with something else? > On Jun 24, 2020, at 9:41 AM, Roy Smith wrote: > > Thank you for reminding me that fixing this has been on my list > <https://github.com/roysmith/spi-to

Re: [Cloud] Need help with selectize.js

Thank you for reminding me that fixing this has been on my list for a while. My CSP-fu is weak. As I understand it, all I need do is: https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css"; - integrity="sha384-ggOyR0iXCbM

Re: [Cloud] Screen sessions

When you ssh into the bastion host, make sure you use the "-t" option to ssh. That forces a pseudo-terminal to get allocated. I have no clue why that's not the default. > On Jun 23, 2020, at 10:47 AM, Isaac Johnson wrote: > > I'm interested in running some long-ish scripts that loop through

[Cloud] Need help with selectize.js

I'm mostly a python guy, who only dabbles in javascript when I can't avoid it :-) I've got a django-based tool (https://tools.wmflabs.org/spi-tools-dev/spi/ ) which uses selectize.js. For simple cases, everything works fine. But, if I click on the

Re: [Cloud] Workflow for kubernetes?

rom a container, so I'd not encountered that issue. > Overall, I'm getting that this is a clash with tmux keybindings, right? > > On a task, we can poke at the issue a bit, and maybe record a solution, > either local to your tool account or in general. > > On 2

Re: [Cloud] Workflow for kubernetes?

using the connect via SSH? I might be able to recreate the issue and figure > out a fix. > -- > Brooke Storm > SRE > Wikimedia Cloud Services > bst...@wikimedia.org <mailto:bst...@wikimedia.org> > IRC: bstorm_ > > > On 2/21/20 7:40 AM, Roy Smith wrote: >>

[Cloud] Workflow for kubernetes?

The bastion hosts don't have python3.7, and the k8s hosts don't have ssh, so "git pull" and "git push" don't work. Which means I need to keep flipping back and forth between the two during development. Is there some way to get ssh working on the k8s hosts, so I can do everything in one place?

[Cloud] PSA about secrets

I was poking around in /data/project/ just now, looking for examples of how other tools set up their django apps. I was surprised (well, only a little) to discover that there's a few world-readable app.py files that have their django_secrets embedded in them. That's not a good idea folks. Sec

[Cloud] cache control on tools-static.wmflabs.org?

Is there a way to set the cache-control header for files served by tools-static.wmflabs.org ?___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimed

[Cloud] etcd has no leader?

I just got the following error. I reran the command and it worked the second time. My limited experience with etcd says this is not a good thing, so figured you'd want to know about it. > tools.spi-tools@tools-sgebastion-08:~/www/python/src$ webservice > --backend=kubernetes python3.7 rest

Re: [Cloud] What's my URL?

does say "The tool name is > used as part of the URL for the tool's webservice." > > On Sun, 12 Jan 2020, 05:47 Roy Smith, <mailto:r...@panix.com>> wrote: > Never mind, I found it by trial-and-error. It's > https://tools.wmflabs.org/spi-tools/ <ht

Re: [Cloud] What's my URL?

Never mind, I found it by trial-and-error. It's https://tools.wmflabs.org/spi-tools/ <https://tools.wmflabs.org/spi-tools/> But, is that documented anywhere in the wiki? > On Jan 12, 2020, at 12:35 AM, Roy Smith wrote: > > I've got a django-based tool that I was p

[Cloud] What's my URL?

I've got a django-based tool that I was previously running in test mode directly on the bastion hosts using runserver. Now I'm trying to move that over the kubernetes for production. I've got things to the point where I can bring up a pod and see the server start by watching uwsgi.log: > SIGI

Re: [Cloud] Searching this list?

https://phabricator.wikimedia.org/T242520 <https://phabricator.wikimedia.org/T242520> > On Jan 11, 2020, at 1:39 PM, RhinosF1 - wrote: > > I can't see there being an issue. Maybe create a phab task for it? > > RhinosF1 > > On Sat, 11 Jan 2020 at 17:59, R

[Cloud] Searching this list?

As far as I can tell, there's currently no way to search this list other than month-by-month via the pipermail archive page ? Not sure what the process is here, but I gather (from T193572 ) that each list h

Re: [Cloud] Adding SSH key to wikitech?

to LDAP. > > On Sat, 11 Jan 2020 at 15:30, Roy Smith <mailto:r...@panix.com>> wrote: > OK,thanks. It's possible I uploaded them long ago for a different project I > was working on, but I don't recall seeing this page before. Are there other > interfaces that m

Re: [Cloud] Adding SSH key to wikitech?

Kok11KTEo0gdDcIEWFyFJAGmsloCPhQiceU2xa0x6OigzxHSufzoNZyNpW+lxDEOxU8fJVMJ8NlVpEH13F5h4dztIAvwvOftE/UZRGfKI1TWzYrmY+YerZUZJF3o9VA0y3D9IE76OFg2BqQpcQh7KbSF0qemM7mFJWdSdMnEbo1AnrX7jxZ+mz0Cdh > RoySmith/wikimedia > > I get the same error if I try to add a duplicate key. > > On Sat, 11 Jan 202

Re: [Cloud] Adding SSH key to wikitech?

> On Jan 11, 2020, at 6:34 AM, Alex Monk wrote: > > Wikitech writes it into LDAP where it'll get read when you SSH into things. > > Please paste your public key here so we can have a look. I've got two different keys I've tried. Both get the same result on wikitech, but work fine in Phab. Pa

[Cloud] Adding SSH key to wikitech?

I'm working my way through the instructions at https://www.mediawiki.org/wiki/Gerrit/Tutorial/tl;dr . I'm up to adding my SSH key to wikitech. I go to https://wikitech.wikimedia.org/wiki/Special:NovaKey

Re: [Cloud] varbinary fields in database?

> On Dec 13, 2019, at 5:04 AM, Jaime Crespo wrote: > > Hi, Roy, > > I will answer assuming you are talking about wikireplicas/WMF production, or > the default Mediawiki installation Yes. Specifically, the instance you get when you run "sql enwiki" from tools-sgebastion-08. > Mediawiki is pl

[Cloud] varbinary fields in database?

Fields such as user.user_name, are varbinary. I'm writing this field as part of a json object in python; json.dumps() requires a string, not bytes, so I need to know the encoding. I assume these are all utf-8? ___ Wikimedia Cloud Services mailing list

Re: [Cloud] toolforge and jupyter?

I figured it out. I can make a directory that's world writable, and then push my data files to there. Seems like a strange way to do it, but I'm not seeing anything cleaner. > On Dec 7, 2019, at 2:12 PM, Roy Smith wrote: > > Hmmm. OK, I see my paws directory mounted on

Re: [Cloud] toolforge and jupyter?

.paws user needs to be able to read and write to the files and > directories there. > > Cheers, > > Chico Venancio > > Em sáb, 7 de dez de 2019 14:27, Roy Smith <mailto:r...@panix.com>> escreveu: > I want to use jupyter to do some data visualization. I found PAWS, bu

[Cloud] toolforge and jupyter?

I want to use jupyter to do some data visualization. I found PAWS, but I can't access my toolforge directory: > FileNotFoundError: [Errno 2] No such file or directory: > '/home/roysmith/sock-classifier/jobs/last/job.out' Is there a way to access my toolforge directory from within PAWS? Altern

[Cloud] Cassandra?

Is there a publicly-accessable Cassandra instance? https://wikitech.wikimedia.org/wiki/Cassandra#Getting_a_CQL_shell implies that there is, but cqlsh doesn't seem to exist on tools-sgebastion-08: > $ cqlsh > -bash: cqlsh: comm

Re: [Cloud] Decoding log_params for blocks?

block > > See more in https://www.mediawiki.org/wiki/Manual:Log_actions > <https://www.mediawiki.org/wiki/Manual:Log_actions> > > Kindly, > > On Wed, Dec 4, 2019 at 10:09 PM Roy Smith <mailto:r...@panix.com>> wrote: >> On Dec 4, 2019, at 8:07 PM, Roy Smith

Re: [Cloud] Decoding log_params for blocks?

> On Dec 4, 2019, at 8:07 PM, Roy Smith wrote: > > I also need to figure out how to parse this in Python. Oh, looks like https://pypi.org/project/phpserialize/ <https://pypi.org/project/phpserialize/> can handle that part.___ Wikimedia

Re: [Cloud] Decoding log_params for blocks?

https://www.php.net/manual/en/function.serialize.php > <https://www.php.net/manual/en/function.serialize.php> > [2] https://www.mediawiki.org/wiki/Manual:Logging_table#log_params > <https://www.mediawiki.org/wiki/Manual:Logging_table#log_params> > On Wed, Dec 4, 2019 at 9:50 PM Ro

[Cloud] Decoding log_params for blocks?

Is there a description somewhere of what the log_params mean in this query? MariaDB [enwiki_p]> select log_params from logging_logindex where log_namespace = 2 and log_title = 'RoySmith-testing' and log_type = 'block' order by log_timestamp asc; +-

[Cloud] /util/arch and /usrlbin//now missing?

When I run qstatus on tools-sgebastion-08, I get the following errors: (env) utils $ qstatus /usr/bin/qstatus: line 299: /util/arch: No such file or directory /usr/bin/qstatus: line 300: /utilbin//now: No such file or directory Waiting jobs for user: roysmith job-ID # name

Re: [Cloud] protobufs?

Thanks again for the additional info. I've been doing more reading about JSON Schema, and am growing fond of the idea. The idea of schema-driven fuzz testing with https://pypi.org/project/hypothesis-jsonschema/ is pretty cool.

Re: [Cloud] protobufs?

read some WIP documentation about how > we use JSONSchema here > <https://wikitech.wikimedia.org/wiki/Event_Platform/Schemas>. > > > > > > > On Fri, Nov 22, 2019 at 10:14 PM Roy Smith <mailto:r...@panix.com>> wrote: > I'm starting to look

[Cloud] protobufs?

I'm starting to look at some machine learning projects I've wanted to do for a while (ex: sock-puppet detection). This quickly leads to having to make decisions about data storage formats, i.e. csv, json, protobufs, etc. Left to my own devices, I'd probably use protos, but I don't want to be s

[Cloud] Intermittent hangs on tools-sgebastion-07?

The last couple of days, I've been having problems with interactive ssh into login.tools.wmflabs.org . Every so often (multiple times an hour, at least), my connection will hang for a few seconds. Sometimes more like 10-15 seconds. I connect from my home MacOS

[Cloud] Unit testing with mwclient?

How do people write unit tests for python mwclient apps? Is there some sort of API mock/fake/stub/whatever that people run tests against? ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https:/

[Cloud] Reverting phabricator vandalism?

A rogue user vandalized a bunch of tasks in phabricator, including several that I opened. I see that account has now been disabled, but is there some easy way to revert the damage they did to my tasks? Or do I need to laboriously edit each task a

[Cloud] Category hidden flag with mwclient?

I want to use mwclient to explore categories. I can get the list of all categories for a page , but I don't see how to limit that to just non-hidden cats. At the API level

Re: [Cloud] Finding tags in wiki text?

. > On Feb 10, 2018, at 3:28 PM, Roy Smith wrote: > > I want to write a tool which checks that you are consistent about ordering of > tags. For example, if I did: > >> The sky is blue. >> The grass is green. > > This would get rendered as: > >> The

[Cloud] Finding tags in wiki text?

I want to write a tool which checks that you are consistent about ordering of tags. For example, if I did: > The sky is blue. > The grass is green. This would get rendered as: > The sky is blue.[1][2] The grass is green.[2][1] When what you want is: > The sky is blue.[1][2] The grass is g

[Cloud] Tmux tab completion?

On login.tools.wmflabs.org , tab completion works for most things on the bash command line, but not tmux. Is there something special I need do to enable tmux tab completion? ___ Wikimedia Cloud Services mailing list Cl

Re: [Cloud] revision.rev_text_id is always 0?

only valid for the underlying > database, you do not have access to that as a labs user - you just > have security-sanitised views. rev_text_ids are not useful to you as > you cannot access revision texts via the DBs - you must go through the > API. > > On 1 October 2017 at 04:31, R

Re: [Cloud] revision.rev_text_id is always 0?

via the DBs - you must go through the > API. > > On 1 October 2017 at 04:31, Roy Smith wrote: >> I’ve been exploring the enwiki database. I can find the page row for >> [[Iron]] >> >> MariaDB [enwiki_p]> select page_title from page where

[Cloud] revision.rev_text_id is always 0?

I’ve been exploring the enwiki database. I can find the page row for [[Iron]] > MariaDB [enwiki_p]> select page_title from page where page_id = 14734; > ++ > | page_title | > ++ > | Iron | > ++ It looks like it has the right number of revisions: > Maria

Re: [Cloud] TFA and Apple Touch ID?

> On Sep 30, 2017, at 3:49 PM, Michael Schönitzer > wrote: > > I agree and even worse: if your password gets stolen you can change it but > you cant change your fingerprint. Actually, that’s not true. The problem is, I can only change it 9 times :-) __

[Cloud] TFA and Apple Touch ID?

What’s the current best practice for auth on ToolForge? I have a passphrase on my public ssh key. I’ll be accessing toolforge from my MacBook which is protected with Apple’s Touch ID fingerprint scanner. I’ll be nailing up a tmux session. So, most of the time, there will be an active ssh sess

Re: [Cloud] [Labs-l] Getting toolforge access?

Thanks. > On Sep 28, 2017, at 11:12 AM, Bryan Davis wrote: > > On Thu, Sep 28, 2017 at 5:36 AM, Roy Smith wrote: >> I applied for a toolforge account about a week ago and haven’t heard >> anything. I’m not sure if I used the wrong process, or if I should just be >

<    1   2