[Cloud] Re: Tunnel to Trove instance?

There's no fundamental reason why this can't work. I've done similar things: > ssh -t dev.toolforge.org -L 3306:tools.db.svc.wikimedia.cloud:3306" Sometimes it takes a few tries to find the right ssh incantation, but port tunneling is definitely something that works. One thing you might try

[Cloud] Re: Update your tool's code to prepare for temporary accounts

Is it possible this is what's been causing https://phabricator.wikimedia.org/T318479? > On May 21, 2024, at 8:55 AM, Szymon Grabarczuk > wrote: > > Hello everyone, > > I am reaching out on behalf of the Wikimedia Foundation Trust and Safety > Product

[Cloud] Horizon: pause vs suspend vs shelve?

I have a VPS instance that I think can be shut down. What I want to do is shut it down in a way that I can restart it again without losing anything, but not yet commit to deleting it. The idea is to leave it in that state for a while just to make sure there's no dependencies on it that i've

[Cloud] Expired certificate?

Poking around on my debian bookworm instance, I found /usr/local/share/ca-certificates/wmf_ca_2017_2020.crt, which looks like an expired SSL certificate: > Certificate: > Data: > Version: 3 (0x2) > Serial Number: > 9f:14:76:9e:ea:f4:18:c3 > Signature

[Cloud] Re: Installing elasticsearch on a VPS instance?

Yup, that did it. I resized it to g3.cores1.ram2.disk20 and now I’m running. Thanks. > On Dec 19, 2023, at 10:54 AM, Francesco Negri wrote: > > On Tue, Dec 19, 2023 at 4:41 PM Roy Smith wrote: >> >> I installed per the instructions at >> https://opensea

[Cloud] Re: Installing elasticsearch on a VPS instance?

kworm instance I just spun up for this. The only configuration I’ve done was to increase vm.max_map_count per their recommendation: > $ cat /proc/sys/vm/max_map_count > 262144 Any ideas? > On Dec 18, 2023, at 6:08 AM, Francesco Negri wrote: > > On Sun, Dec 17, 2023 at 10:04

[Cloud] Re: Installing elasticsearch on a VPS instance?

17, 2023 at 10:04 PM Roy Smith wrote: >> I want to install elasticsearch on a VPS instance. I’m guessing this >> involves puppet, but can’t figure out the details. The instructions at >> https://wikitech.wikimedia.org/wiki/Help:Puppet don’t match what I’m >>

[Cloud] Fwd: Installing elasticsearch on a VPS instance?

Trying this again without the screenshot, which got stuck in the filters. > Begin forwarded message: > > From: Roy Smith > Subject: Installing elasticsearch on a VPS instance? > Date: December 17, 2023 at 4:00:08 PM EST > To: Wikimedia Cloud Services general discussion and

[Cloud] Re: Webservice logs?

though, as usual, it's > really hard to promise anything). I expect that project will be tracked in > subtasks of T127367 <https://phabricator.wikimedia.org/T127367> once there's > anything beyond a very rough idea. > > Taavi > > On Thu, Oct 5, 2023 at 9:10 P

[Cloud] Webservice logs?

I see T336057 was just closed. Looking at the docs , I'm unclear how this works. The docs say ", the output from the webservice command is stored by the Toolforge Kubernetes infrastructure as long as the web

[Cloud] Re: [Cloud-announce] Toolforge Kubernetes upgrade on 2023-04-03 (new date: 2023-04-10)

Just to make sure I'm clear, the downtime announced yesterday is still happening? > On Mar 30, 2023, at 6:42 AM, Arturo Borrero Gonzalez > wrote: > > On 3/28/23 00:13,

[Cloud] Redis vulnerabilities

There's a couple of new vulnerabilities for redis that were just published, which people should be aware of: https://nvd.nist.gov/vuln/detail/CVE-2023-28858 https://nvd.nist.gov/vuln/detail/CVE-2023-28859

[Cloud] Does an account rename invalidate OAuth keys?

I've got a tool user DKY-Tools-Bot, that I want to rename to be DYKToolsBot. I've got two sets of OAuth keys: https://meta.wikimedia.org/wiki/Special:OAuthListConsumers/view/f1a1f081a16a1261b0ee99e61e172a5f

[Cloud] Re: ssh to tool account?

in phab). > On Feb 15, 2023, at 9:50 AM, Taavi Väänänen wrote: > > On 2/15/23 16:37, Roy Smith wrote: >> Am I just doing something wrong, or is toolforge set up to disallow that? > > Yes, all of Cloud VPS (including Toolforge) is configured to only load SSH > keys fr

[Cloud] ssh to tool account?

I'm trying to simplify my life by avoiding having to log in to my own account on a toolforge bastion and then become my tool. I tried dropping my public key into $HOME/.ssh/authorized_keys for my tool, but that doesn't work: > $ ssh -t tools.dyk-to...@dev.toolforge.org >

[Cloud] Re: Scratch database instance?

string, but to compute a hash, store that as a separate column, and index the hash. Which feels like I'm working for the database instead of the other way around. > On Jan 25, 2023, at 3:52 PM, Daniel Zahn wrote: > > On Wed, Jan 25, 2023 at 12:33 PM Roy Smith wrote: >> >> Than

[Cloud] Re: Scratch database instance?

Thanks. This field is intended to hold a wikipedia page title (including the namespace). Would 255 be long enough to hold any legal title? > On Jan 25, 2023, at 11:53 AM, Daniel Zahn wrote: > > Yea, it's variable length but it still wants you to give it a maximum length.

[Cloud] Scratch database instance?

I just discovered that some code I've been working on, which passes all its unit tests against sqllite, fails when I run it against tools.db.svc.wikimedia.cloud. It's apparently due differences in the two drivers (see for example, this bug ).

[Cloud] Re: Migrating to a newer VPS image?

Will do, thanks. > On Jan 3, 2023, at 10:56 AM, Bryan Davis wrote: > > On Tue, Jan 3, 2023 at 8:54 AM Roy Smith wrote: >> >> Putting it on a volume sounds like a plan, thanks. For now, there wasn't a >> whole lot of data, so scp was good enough. >> >&g

[Cloud] Re: Migrating to a newer VPS image?

ittle >> quicker? (i.e., mount it on the old instance, copy the data to it, remount >> it on the new instance, copy the data off) >> >> >> >> Kind regards, >> Sammy >> >> >> On Tue, 3 Jan 2023, 10:10 Roy Smith, wrote: >> >>&

[Cloud] Re: Proposing a Tool sweep

It seems like all of those checks could be automated. Have we reached the point where "source code published somewhere" is synonymous with "in a publicly accessible git repo"? > On Dec 29, 2022, at 4:02 AM, Kunal Mehta wrote: > > Hi everyone, > > tl;dr:

[Cloud] Need ideas for debugging wedged web service (toolforge)

spi-tools and spi-tools-dev both occasionally get wedged. HTTP requests just hang and eventually time out with a 50x. Nothing gets logged in either my django application log, or in uwsgi.log. If I restart the service, things are fine until it happens again. Any ideas how I can get better

[Cloud] My logs turned into .nfs files, what's going on?

Yesterday, I reported some log messages in T318479 : logs/django/django.log.2022-09-27:2022-09-28 16:43:39,873 [76e999afc82c10fb99b6c9bf76448d1a] INFO tools_app.middleware: IndexView() logs/django/django.log.2022-09-27:2022-09-28 16:59:18,903

[Cloud] Problems with Redis?

Is there an issue with the Toolforge redis server? Every so often, I get connection timeouts: django.log.2022-09-10:2022-09-11 18:31:17,239 [0b5fd61ed67203bfe290ca99fcb52e1e] ERROR tools_app.redis: Redis ConnectionError: Error while reading from socket: (110, 'Connection timed out')

[Cloud] Password change on wikitech?

I couldn't log into wikitech or horizon this morning. A password reset fixed the problem. Did something happen to invalidate passwords? ___ Cloud mailing list -- cloud@lists.wikimedia.org List information:

[Cloud] How to get django to serve robots.txt in toolforge?

I want to serve a robots.txt file from my django app (spt-tools). My settings.py file has: STATIC_ROOT = f'{WWW_DIR}/static/' so when I run "manage.py collectstatic", robots.txt ends up in $HOME/www/static/robots.txt. It needs to be one directory level up from there. I can think of

[Cloud] Custom puppet class?

I want to set up a custom rsyslog config on my spi-tools VPS instance. I know what I want to end up with, but I'm trying to get it puppetized. It's not really clear what I need to do. Do I really need to set up my own standalone puppetmaster

[Cloud] Re: Connecting from Toolforge to VPS?

On May 25, 2022, at 3:03 PM, Roy Smith wrote: > > OK, got that sorted out and tested with netcat, thanks. > > Now I just need to sort out some hand-to-hand combat with rsyslog configs > <https://serverfault.com/questions/1101795/messages-sent-to-unix-socket-get-logged-but-not

[Cloud] Re: Connecting from Toolforge to VPS?

stated, all statements from this account are made in a > volunteer capacity, and may not reflect the views of the Wikimedia Foundation. > > This message contains confidential information and is intended only for the > individual named. If you are not the named addressee, you should not > dis

[Cloud] Connecting from Toolforge to VPS?

I want to send data from a process running on toolforge to a VPS host. I tried the obvious: On the VPS host (puppet-test.spi-tools.eqiad1.wikimedia.cloud): > $ nc -4 -l -p 23001 On tools-sgebastion-11: > echo foo | nc -v -4 puppet-test.spi-tools.eqiad1.wikimedia.cloud 23001 > nc: connect to

[Cloud] checkuser via a tool?

I've been experimenting with adding some checkuser functionality to spi-tools, and scratching my head over why I'm getting permissiondenied errors. I think I finally figured it out. It looks like my OAuth consumer

[Cloud] Re: Kubernetes-based jobs engine -- how to use Python virtual environments?

I second that. I used to not worry too much about doing it right, but eventually got bitten by my sloppyness. The problem seems to be that the paths which get embedded in your venv executables will be different, which leads to hard-to-debug problems. > On Apr 6, 2022, at 10:12 AM,

[Cloud] Redis eviction stats?

I'm thinking about using the Toolforge Redis instance for a persistent data store. For the data I want to store, it's OK if there's occasional evictions due to memory exhaustion, but only for fairly small values of "occasional". So, I'm curious what the situation is for this particular

[Cloud] Re: [Cloud-announce] [IMPORTANT] Announcing Toolforge Debian Stretch Grid Engine deprecation

From my perspective of a Toolforge user, one of the issues I see is that it's often not clear how to map the "friendly command line interface" into concepts I already understand about the lower level tools. For example, the webservice script does some useful stuff. But, it wasn't clear

[Cloud] django with oauth?

I'm running Django 3.1 and social-auth-app-django 3.1.0 so I can authenticate against MediaWiki. Everything works, but want to start moving toward more modern releases. If I upgrade social-auth-app-django to anything newer, I get an exception in the authentication flow: Direct assignment to

[Cloud] Re: elasticsearch.exceptions.UnsupportedProductError

ersions is not permitted on WMCS. > > -Andrew > > On 1/10/22 8:49 PM, Roy Smith wrote: >> Never mind, I found it. The problem is that newer versions of elasticsearch >> (> 7.14?) do over-eager version checking as described here >> <https://stackoverflow.c

[Cloud] Re: elasticsearch.exceptions.UnsupportedProductError

to https://opensearch.org/ <https://opensearch.org/> which looks like it's working. Apparently this affects multiple language bindings (Python, Ruby, JS, maybe others). > On Jan 10, 2022, at 9:35 PM, Roy Smith wrote: > > I'm exploring using elasticsearch in Toolforge.

[Cloud] elasticsearch.exceptions.UnsupportedProductError

I'm exploring using elasticsearch in Toolforge. I'm running Python 3.7, with elasticsearch==7.16.2 installed. When I run: def main(): es = Elasticsearch(['elasticsearch.svc.tools.eqiad1.wikimedia.cloud:80']) print(es.info()) I get: > elasticsearch.exceptions.UnsupportedProductError:

[Cloud] How many kubernetes jobs is too many?

I'm starting on a project which will process every edit comment in the enwiki dumps using k8s jobs on toolforge. There's 736 enwiki-20211201-pages-meta-history*.xml*bz2 files. Would kicking off 736 jobs (one per file) be a reasonable thing to do from the resource consumption point of view, or

[Cloud] Re: Source command not working with zsh

ar why the scripts > would fail on the grid, and why they would fail without outputing anything. > > On Tue, Nov 16, 2021 at 9:43 PM Roy Smith <mailto:r...@panix.com>> wrote: > Submit a job that does: > >> ls -l /usr/bin/zsh > > and see what it says. >

[Cloud] Re: Source command not working with zsh

Submit a job that does: > ls -l /usr/bin/zsh and see what it says. > On Nov 16, 2021, at 9:37 PM, Huji Lee wrote: > > Is it really that zsh is not available on the grid, and the grid tries to > replicate my environment first and reaches the "exec zsh" command and falls > apart somehow?

[Cloud] Re: Source command not working with zsh

This is really good advice. Any time you've got a process being run by some tool on your behalf (cron, initd, remote job execution, etc), you're running in an alien environment. You get so used to things "just working" when you run them interactively, you forget how much of your carefully

[Cloud] Re: Source command not working with zsh

is valid on any possible execution machine. And pre-creating the file mode 0666 becomes more likely to be necessary. > On Nov 12, 2021, at 1:16 PM, Roy Smith wrote: > > No matter how screwed up your env, path, choice of shell, output redirection, > etc are, that's pretty muc

[Cloud] Re: Source command not working with zsh

When I can't figure out WTF is going on with a shell script, I put something like this in the front of it: /usr/bin/env > /tmp/this-is-my-environment No matter how screwed up your env, path, choice of shell, output redirection, etc are, that's pretty much guaranteed to dump some useful

[Cloud] toolforge down?

https://spi-tools.toolforge.org/spi/ and https://xtools.wmflabs.org/ are loading extremely slowly. Actually, on both, it looks like you eventually get the HTML, but none of the statics (CSS, images, etc) load.

[Cloud] Re: Accessing database replicas from VPS host?

Never mind. PEBKAC. > On Oct 7, 2021, at 4:19 PM, Roy Smith wrote: > > I have a VPS instance, spi-tools-host-1 > <https://horizon.wikimedia.org/project/instances/43d3e9f2-b7d5-4a51-92b6-6a636839d66f/>, > from which I want to query the database replicas. Followin

[Cloud] Accessing database replicas from VPS host?

I have a VPS instance, spi-tools-host-1 , from which I want to query the database replicas. Following the instructions at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database

[Cloud] Re: Python upgrade on the Toolforge

> On Sep 28, 2021, at 12:56 PM, Huji Lee wrote: > > Do you have any links to share about building python from source? First, download the source tarball. If I were doing this today, I'd use Python 3.9.7, which is at https://www.python.org/downloads/release/python-397/

[Cloud] Re: Python upgrade on the Toolforge

I don't know if it's generally available yet, but I've been using tools-sgebastion-11, which is running Debian Buster with Python 3.7. On the other hand, I don't actually use the system Python. Quite a while ago, before the Buster machine was available, I built Python 3.7 from source and I've

[Cloud] Re: Validating multiple usernames?

e canonical > form of page titles is with underscores. > > No this hasn't ever confused anyone or caused me any problems, why do you ask? > > ACN > > On Thu, Sep 9, 2021 at 10:56 AM Roy Smith wrote: >> >> That can't be right. I think you meant, "Reduc

[Cloud] Re: Validating multiple usernames?

That can't be right. I think you meant, "Reduce any runs of multiple underscores to a single SPACE" and then "Trim any leading or trailing spaces" > On Sep 6, 2021, at 12:15 AM, Bryan Davis wrote: > > * Replace all whitespace characters with underscores (`_`) > * Reduce any runs of multiple

[Cloud] Re: Validating multiple usernames?

, the entries aren't guaranteed to be in the same order as the input (despite the fact that the python mwclient goes out of its way to present it as an OrderedDict), and the output keys aren't guaranteed to match the input keys. > On Sep 5, 2021, at 3:18 PM, Roy Smith wrote: > > I'm

[Cloud] Re: Validating multiple usernames?

"missing" : "" } ] }, "batchcomplete" : "" } I'm assuming the entries in the returned "users" list are guaranteed to be in the same order as the input parameters? I can't find anyplace that says this, but it seems lo

[Cloud] Re: Validating multiple usernames?

ntext since it can't map it to a userid. I can work around this, but mentioning it for the sake of some poor developer searching the archives N years from now trying to figure it out :-) > On Aug 19, 2021, at 6:21 PM, Bryan Davis wrote: > > On Thu, Aug 19, 2021 at 4:04 PM Roy Smith wrote:

[Cloud] Re: Validating multiple usernames?

Ah, cool. That's exactly what I was looking for, thanks. > On Aug 19, 2021, at 6:21 PM, Bryan Davis wrote: > > On Thu, Aug 19, 2021 at 4:04 PM Roy Smith wrote: >> >> I've got a tool which parses sockpuppet investigation (SPI) pages and does >> some analysis.

[Cloud] Validating multiple usernames?

I've got a tool which parses sockpuppet investigation (SPI) pages and does some analysis. One of the steps is I need to validate that all of the usernames found in the SPI report are valid. I do that by sequentially calling usercontribs on each name with uclimit=1 and seeing if I get a

[Cloud] Re: Hibernating a VM?

ly your server is in 'shelving' status, so it sholud be waiting for it > to shut off, once that happens it will > consider it as shelved. > If it's still running in a while feel free to open a ticket so we do some > debugging. > > Cheers! > > > On 06/07 09:50, Roy Smith wro

[Cloud] Re: Hibernating a VM?

Looking in https://horizon.wikimedia.org/project/instances/43d3e9f2-b7d5-4a51-92b6-6a636839d66f/ , I also see "Shelve Instance"; what does that do? I'm actually kind of curious about that last one because

[Cloud] Hibernating a VM?

I've got a VM (spi-tools-host-1) that I'm not actively using right now. I could shut it down to release the CPU resources, but I want to be able to start it back up again at some point in the future with the same configuration. On AWS I would just back up all the storage and shut down the VM.

Re: [Cloud-announce] [Cloud] cloud-vps maintenance at 14:00 UTC

I'm getting timeouts and 502's on both https://spi-tools.toolforge.org/ and https://spi-tools-dev.toolforge.org/ . Also: ssh: connect to host dev.tools.wmflabs.org port 22: Network is unreachable Could the VPS work be

Re: [Cloud] [Cloud-announce] cloud-vps maintenance at 14:00 UTC

I'm getting timeouts and 502's on both https://spi-tools.toolforge.org/ and https://spi-tools-dev.toolforge.org/ . Also: ssh: connect to host dev.tools.wmflabs.org port 22: Network is unreachable Could the VPS work be

[Cloud] NFS bandwidth to VPS nodes?

I'm exploring various ways of working with the XML data dumps on /publib/dumps/public/enwiki. I've got a process which runs through all of the enwiki-20210301-pages-articles[123456789]*.xml* files in about 6 hours. If I've done the math right, that's just about 18 GB of data, or 3 GB/h, or 8

Re: [Cloud] Publishing out of a VPS node?

rk > Failing that, what's the name of your instance and project? I can probably > sort it out for you > > On Sun, 18 Apr 2021 at 13:30, Roy Smith <mailto:r...@panix.com>> wrote: > Hmmm, it turns out, this doesn't work. Apache fails to start with: > > AH00534: apache2:

Re: [Cloud] Publishing out of a VPS node?

mkdir ~/public_html/ > > copy files to public_html as you already mentioned > > then click in Horizon to create a proxy, select a name and the > instance it should point to > access data under that name + /~username/ > > On Sat, Apr 17, 2021 at 3:27 PM Roy Smith wrote: &

[Cloud] Publishing out of a VPS node?

I've computed some data on a VPS node that I want to show to people. Is there some quick and dirty way to publish a file so it's visible to the outside? I'm thinking something along the lines of a public_html directory. ___ Wikimedia Cloud Services

Re: [Cloud] [Cloud-announce] New Wikireplicas available, timeline update, and Quarry migration

or should I create a new project? > On Mar 31, 2021, at 10:35 PM, Roy Smith wrote: > > Thanks for looking into this. I tried this again a little later, and it ran > fine. Odd that the amount of memory used depends on the number of rows. I > would expect it would stream res

Re: [Cloud] [Cloud-announce] New Wikireplicas available, timeline update, and Quarry migration

>> On Mar 31, 2021, at 5:18 PM, Roy Smith > <mailto:r...@panix.com>> wrote: >> >> I'm just playing around on tools-sgebastion-08. I can dump the first 1 >> million image names about half a minute: >> >>> tools.spi-tools-dev:xw

Re: [Cloud] [Cloud-announce] New Wikireplicas available, timeline update, and Quarry migration

I'm just playing around on tools-sgebastion-08. I can dump the first 1 million image names about half a minute: > tools.spi-tools-dev:xw-join$ time mysql --defaults-file=$HOME/replica.my.cnf > -h commonswiki.web.db.svc.wikimedia.cloud commonswiki_p -N -e 'select > img_name from image limit

Re: [Cloud] [Cloud-announce] New Wikireplicas available, timeline update, and Quarry migration

Is it feasible to do a log analysis of the database servers to find out what tools are (were?) using cross-wiki joins? At least that would ensure that all the tool owners could be contacted directly to make sure they know this is happening. > On Mar 31, 2021, at 3:46 PM, Joaquin Oltra

Re: [Cloud] Install emacs 25 on bastion hosts?

Should I open a phab ticket for requests like this? > On Mar 19, 2021, at 10:02 PM, Roy Smith wrote: > > Not sure if this is the right place to ask, but would it be possible to > install emacs-25 on the bastion hosts? Apparently, the python mode in > emacs-25 knows about the

[Cloud] Install emacs 25 on bastion hosts?

Not sure if this is the right place to ask, but would it be possible to install emacs-25 on the bastion hosts? Apparently, the python mode in emacs-25 knows about the new python-3.5 asyncio keywords (async def, etc), but emacs-24 doesn't. One of these looks like the right thing:

Re: [Cloud] tmpfs for tool account?

I should have mentioned I'm talking about tools-sgebastion-08. > On Feb 19, 2021, at 2:28 PM, Roy Smith wrote: > > My personal account has an entry in /run/user: > > drwx-- 4 roysmith wikidev 80 Jan 14 14:04 /run/user/17980 > > but not my tool account. Is it p

[Cloud] tmpfs for tool account?

My personal account has an entry in /run/user: drwx-- 4 roysmith wikidev 80 Jan 14 14:04 /run/user/17980 but not my tool account. Is it possible to set one up for tool accounts? ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org

Re: [Cloud] X_REQUEST_ID?

oysmith/haproxy-xuri>). The haproxy folks didn't pick up my patch, but I see they did eventually implement their own version. > On Feb 16, 2021, at 11:54 AM, Bryan Davis wrote: > > On Sun, Feb 14, 2021 at 8:43 AM Roy Smith wrote: >> >> I'm running a django app

[Cloud] X_REQUEST_ID?

I'm running a django app under toolforge. I see there's already a X_REQUEST_ID: e9ec1899b09244939d22c2387db3b75a header in the incoming request. That's totally cool, but I don't see it documented in the toolforge docs. is that something the toolforge infrastructure guarantees will be there?

Re: [Cloud] [Cloud-announce] new Cloud VPS feature: attachable block storage

If it's on its own file system, df, rather than du, is your friend. > On Feb 5, 2021, at 11:21 PM, Andrew Bogott wrote: > > [0] So big that I don't know how big it is because I ran 'du' and gave up > after it sat there for 40 minutes ___ Wikimedia

[Cloud] {dev,login}.tools.wmflabs.org ssh down?

Is there a problem with SSH on {dev,login}.tools.wmflabs.org ? I can ping both of those hosts, but SSH is not responding for me. ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly

Re: [Cloud] URL length limit for toolforge?

tml#more_itertools.time_limited> which I'm sure will come in handy here. > On Dec 16, 2020, at 10:50 PM, Bryan Davis wrote: > > On Wed, Dec 16, 2020 at 6:04 PM Roy Smith <mailto:r...@panix.com>> wrote: >> >> The the following (absurdly long) URL: >> >>

[Cloud] URL length limit for toolforge?

The the following (absurdly long) URL: >

Re: [Cloud] password for thanos?

Thanks. Backing up a step, what I'm looking to do is build some kind of performance and monitoring dashboard for my tool. From what you say, maybe Thanos is not the right thing for that? > On Dec 15, 2020, at 12:25 PM, Bryan Davis wrote: > > On Tue, Dec 15, 2020 at 10:18 AM Roy Smi

[Cloud] password for thanos?

I'm trying to connect to https://thanos.wikimedia.org/ . It's asking for my "Wikimedia Developer Username" and password, but I'm not sure which credentials it wants. Is this the same username I use to log into my shell account on dev.tools.wmflabs.org

Re: [Cloud] [Cloud-announce] tools bastion downtime on Monday, 2020-10-19 at 14:00 UTC

0/15/20 2:47 PM, Roy Smith wrote: >> There was some talk a while ago about an upgrade which would reduce NFS >> latency between the bastions and the k8s cluster. Is that what this is? > > I'm not sure what that refers to -- maybe upgrading the NFS server's network > connect

Re: [Cloud] [Cloud-announce] tools bastion downtime on Monday, 2020-10-19 at 14:00 UTC

There was some talk a while ago about an upgrade which would reduce NFS latency between the bastions and the k8s cluster. Is that what this is? > On Oct 15, 2020, at 3:11 PM, Andrew Bogott wrote: > > On Monday at 14:00 UTC I'll be moving both toolforge bastions to the new > storage

Re: [Cloud] Python3.7

would you set up your development and production environments, given the current state of Toolforge? > On Oct 7, 2020, at 8:16 PM, Roy Smith wrote: > >> On Oct 7, 2020, at 7:28 PM, Bryan Davis > <mailto:bd...@wikimedia.org>> wrote: >> >> This may actuall

Re: [Cloud] Python3.7

> On Oct 7, 2020, at 7:28 PM, Bryan Davis wrote: > > This may actually be the root of some of your frustrations. Toolforge > is not meant to be a replacement for a development environment on your > local laptop or other server. Hmmm. That's not the impression I got from reading what's

Re: [Cloud] Python3.7

It would be really good if the bastions could be updated to something more modern, so we could have a uniform python infrastructure everywhere. I think I once opened a phab ticket for this, which was closed as some variation on infeasible. I ran into exactly the issue described below myself.

Re: [Cloud] Parsoid APIs?

tra Hernandez > wrote: > > Hi Roy, > > Some responses inline: > > > On Fri, Sep 4, 2020 at 6:41 PM Roy Smith <mailto:r...@panix.com>> wrote: > I know there's been a ton of work done of Parsoid lately. This is great, and > the amount of effort that's gone

[Cloud] Parsoid APIs?

I know there's been a ton of work done of Parsoid lately. This is great, and the amount of effort that's gone into this functionality is really appreciated. It's clear that Parsoid is the way of the future, but the documentation of how you get a Parsoid parse tree via an AP call isI kind of

[Cloud] Logging vis-a-vis WMF privacy policy?

Is there any specific guidance for what's appropriate or not appropriate for my tool to be logging? I'm using OAUTH, so my tool knows the identity of the user. I assume I don't want to be logging that. Or, is it OK, as long as the logs stay within toolforge? For debugging purposes, I'd like

[Cloud] What to do with obsolete oauth consumer keys?

I've created a handful of keys, experimenting the various configs. Now that I've figured out what I need, I have a bunch of old consumers that I don't need any more. Is there anything I should do to revoke them? ___ Wikimedia Cloud Services mailing

Re: [Cloud] Where to look up my oauth tokens?

basic + viewdeleted? Is there such a thing as a "read-only" consumer? For what I want to do, I'll never want my app to do any edits. It would make sense to get a read-only consumer key, if such a thing existed. > On Jul 9, 2020, at 7:29 PM, Roy Smith wrote: > > I've got several oauth t

[Cloud] Where to look up my oauth tokens?

I've got several oauth tokens that I registered for a while ago. I have the actual tokens, but where I can I look up the details like whether I registered them as owner-only, etc? ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org

Re: [Cloud] Toolforge: new domain toolforge.org

I've started getting: > Invalid HTTP_HOST header: 'spi-tools-dev.toolforge.org'. You may need to add > 'spi-tools-dev.toolforge.org' to ALLOWED_HOSTS. I assume that's related to the change? > On Jul 7, 2020, at 6:50 AM, Arturo Borrero Gonzalez > wrote: > > On 2020-07-06 17:59, Arturo

Re: [Cloud] Slow uWSGI logging

https://phabricator.wikimedia.org/T256426 <https://phabricator.wikimedia.org/T256426> > On Jun 25, 2020, at 4:00 PM, YiFei Zhu wrote: > >> Should I open a phab ticket on this? > > Yes please. > > On Thu, Jun 25, 2020 at 2:55 PM Roy Smith wrote: >> >

Re: [Cloud] Slow uWSGI logging

g the data... > > YiFei Zhu > > On Thu, Jun 25, 2020 at 8:12 AM Roy Smith wrote: >> >> https://tools.wmflabs.org/spi-tools/spi/ >> >> To generate a request, click the "Sock Info" button or the "Index" link. >> >> The log file is

Re: [Cloud] Slow uWSGI logging

tool is this? I can probably strace and see if that is the case. > > YiFei Zhu > > > On Wed, Jun 24, 2020 at 10:16 PM Roy Smith wrote: >> >> I'm running a web server with "webservice --backend=kubernetes python3.7". >> As I tail the uwsgi.log file,

[Cloud] Slow uWSGI logging

I'm running a web server with "webservice --backend=kubernetes python3.7". As I tail the uwsgi.log file, requests to my server get logged with very long delays. I just timed one at about a minute and a half between when the request was served (03:07:57 UTC 2020) and when it showed up in the

Re: [Cloud] Need help with selectize.js

g) that will provide some protection against the file being unexpectedly replaced with something else? > On Jun 24, 2020, at 9:41 AM, Roy Smith wrote: > > Thank you for reminding me that fixing this has been on my list > <https://github.com/roysmith/spi-tools/issues/4>

Re: [Cloud] Need help with selectize.js

Thank you for reminding me that fixing this has been on my list for a while. My CSP-fu is weak. As I understand it, all I need do is: https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css; -

Re: [Cloud] Screen sessions

When you ssh into the bastion host, make sure you use the "-t" option to ssh. That forces a pseudo-terminal to get allocated. I have no clue why that's not the default. > On Jun 23, 2020, at 10:47 AM, Isaac Johnson wrote: > > I'm interested in running some long-ish scripts that loop

  1   2   >