Re: [Labs-l] MediaWiki comment storage schema changes coming, may break your SQL queries

[Brad Jorsch (Anomie)]

On Wed, Jun 7, 2017 at 1:46 PM, Bryan Davis  wrote:

> There is a task in the late stages of planning [0] that will change
> how comments are stored in MediaWiki.


There's also a similar task in the late stages of planning[2] for changing
how users are stored in tables such as revision (rev_user+rev_user_text).
That could also use your comments.

[2]: https://phabricator.wikimedia.org/T167246


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] labs-l:如何处理员工违纪问题? 6dkn7m

[Brad Jorsch (Anomie)]

2017-02-15 23:38 GMT-05:00 Maximilian Doerr :

> What the hell is this?
>

Probably a macro virus trying to spread itself, although it's possible it's
just a completely off-topic post (Google Translate makes the text sound
like some workers' rights thing).


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Reimplementing account expiries

[Brad Jorsch (Anomie)]

On Fri, Feb 3, 2017 at 11:28 AM, Jaime Crespo  wrote:

> On Fri, Feb 3, 2017 at 4:59 PM, Brad Jorsch (Anomie) <
> bjor...@wikimedia.org> wrote:
>
>> On Fri, Feb 3, 2017 at 10:49 AM, Maximilian Doerr <
>> maximilian.do...@gmail.com> wrote:
>>
>>> Hey guys, I thought I would throw a quick suggestion in to the mix.  We
>>> frequently encounter situations where labs hits resource limits because of
>>> tools consuming space, either by negligence, or the owner disappeared.
>>>
>>
>> {{citation needed}}, particularly in that this is a problem on Tool Labs
>> specifically.
>>
>
> https://phabricator.wikimedia.org/T132431
> https://phabricator.wikimedia.org/T133321#2979483
>

Those both look like a particular tool using too much space, rather than a
large number of inactive tools adding up.


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Reimplementing account expiries

[Brad Jorsch (Anomie)]

On Fri, Feb 3, 2017 at 10:49 AM, Maximilian Doerr <
maximilian.do...@gmail.com> wrote:

> Hey guys, I thought I would throw a quick suggestion in to the mix.  We
> frequently encounter situations where labs hits resource limits because of
> tools consuming space, either by negligence, or the owner disappeared.
>

{{citation needed}}, particularly in that this is a problem on Tool Labs
specifically.

The only thing along these lines that comes to my mind is the recent
cleanup of old unused Labs instances, which was nothing to do with Tool
Labs.

Ok, and there was concern a while back about *log files* consuming space on
Tool Labs. But that has little to do with inactive users.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] [Labs-announce] [notice] DB replicas user_properties view redaction

[Brad Jorsch (Anomie)]

On Wed, Nov 30, 2016 at 9:14 PM, Huji Lee  wrote:

> Makes no sense to me to exclude data from these tables that is already
> available through MediaWiki API. Why would you hide "gender" here when it
> is accessible through something like [1]?
>

It's also available via the {{gender:}} magic word.

But I don't know why you're complaining about hiding "gender" when it is
still available via the user_properties view, along with "disablemail",
"fancysig", and "nickname".


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] OAuth with RSA key

[Brad Jorsch (Anomie)]

On Wed, Oct 26, 2016 at 1:08 AM, Magog The Ogre 
wrote:

> I tried to set:
> consumerKey = the blacked out key on the screenshot
> consumerSecret = the private RSA key (I deleted the public key while
> testing, so it doesn't appear in the screenshot)
>

consumerSecret is a hash value given to you when you register the consumer
or when you update it with the "Reset the secret key to a new value"
checkbox checked. You can see a screenshot of the output at
https://phabricator.wikimedia.org/F4663097; in that example the secret is
"7086b640b893d53d4236cf8f4769390876a9cbe1".

The hello-world app does not support RSA keys.


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] OAuth with RSA key

[Brad Jorsch (Anomie)]

On Fri, Oct 21, 2016 at 8:56 PM, Magog The Ogre 
wrote:

> Well I tried to do this using the "pre-shared secret key" method described
> at https://tools.wmflabs.org/oauth-hello-world/ and I just get an invalid
> key error, so I'm doing something wrong.
>

Without seeing your actual code, I can't begin to guess just what you might
be doing wrong. Does it work for you if you copy the hello-world code and
put your own credentials in the .ini file?

The contents of that .ini file would be something like this:

agent = Your user agent
consumerKey = 0123456789abcdef0123456789abcdef
consumerSecret = 0123456789abcdef0123456789abcdef01234567


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] OAuth with RSA key

[Brad Jorsch (Anomie)]

On Sun, Oct 16, 2016 at 11:28 PM, Magog The Ogre 
wrote:

> The one example program makes only a vague reference to it (
> https://tools.wmflabs.org/oauth-hello-world/index.php?action=download),
> and the other (https://www.mediawiki.org/wiki/OAuth/For_Developers#PHP_
> demo_cli_client_with_RSA_keys) presupposes that I have preexisting
> libraries installed, for some reason.
>

In any case using RSA keys is going to want you to have some sort of
library installed to deal with using RSA keys, if not a library to handle
all the OAuth stuff itself.

The libraries needed for your second link are included in the OAuth
extension itself. The most relevant bit for your question here is the
OAuthSignatureMethod_RSA_SHA1::build_signature()
method[1] which uses PHP's openssl extension to do the actual signing.[2]

 [1]:
https://phabricator.wikimedia.org/diffusion/EOAU/browse/master/lib/OAuth.php;51cd54d332d1b6c0647f3be699c000833cb9d54e$235-252
 [2]: https://secure.php.net/openssl


> I just want to be able to authenticate the identity of users. Magnus
> Manske has said he doesn't want to maintain TUSC anymore, but I cannot
> figure out how to do this with the documentation provided.
>

Note that using RSA-SHA1 rather than HMAC-SHA1 is in no way required.


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Guidance on how to set up pywikibot cronjobs and utilise oauth

[Brad Jorsch (Anomie)]

On Sat, Mar 19, 2016 at 9:07 AM, billinghurst 
wrote:

> Since I converted my (pywikbot) bot to use oauth authentication, it is
> failing on its cronjob due to not authenticating.
>
> ERROR: OAuth authentication not supported: No module named
> requests_oauthlib
>

It appears in some quick checking that the system-wide
python-requests-oauthlib package is not available on the default precise
hosts, but is on trusty (and tools-login runs on trusty). Try adding "-l
release=trusty" to your job submission.

I also note that python-requests-oauthlib isn't explicitly installed at
all, it's only being pulled in because it's a dependency of python-twitter.
Someone should probably file a task requesting that it be installed
explicitly.

Or you might be able to install it in python's virtualenv for your
bot/account, but don't ask me how to do that because I don't know anything
about it besides that it exists. ;)
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Labs privacy policy questions

[Brad Jorsch (Anomie)]

On Tue, Mar 15, 2016 at 9:08 PM, Platonides  wrote:

> A problem I find with OAuth is that often you don't know at all what it is
> going to do.
>
> So, taking wikimetrics as an example, it says:
> «In order to complete your request, Wikimetrics Website needs permission
> to access information on meta.wikimedia.org on your behalf. No changes
> will be made with your account.»
>
> Which information does it access? Your account name? Your watchlist? The
> checkuser log (supposing you were a CU)?
>

That particular message is used when it will only be able to get some
information about your user account: your username, edit count, whether you
confirmed your email address, whether you're blocked, when your account was
created, what groups your account is a member of, what user rights are
available to your account, what grants are available to the OAuth
application, and (sometimes[1]) your "real name"[2] and email address.
Since the OAuth application isn't being allowed to use the 'read' right, it
won't be able to access much of anything else.

If you'd like to suggest improvements to the message, the messages are
mwoauth-form-description-allwikis-nogrants and
mwoauth-form-description-onewiki-nogrants. You could reply here with
suggestions, although it might be easier to track in Phabricator, or you
could submit a patch yourself with better wording.


[1]: It depends if the OAuth consumer was registered as "Authentication
only, no API access" or "Authentication only with access to real name and
email address via Special:OAuth/identify, no API access".
[2]: MediaWiki can have a "Real name" field in Special:Preferences, but
this is hidden on WMF wikis.

-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] [Labs-announce] abrupt tools login host changes

[Brad Jorsch (Anomie)]

On Sun, Feb 21, 2016 at 5:16 PM, Andrew Bogott 
wrote:

> I've already moved all of the affected hostnames (e.g.
> tools-login.wmflabs.org) so if you get kicked off of the bastion, just
> reconnect and all will be more-or-less as it was.


It seems that the grid doesn't know about the new tools-login, so I'm
getting "host "tools-bastion-05.tools.eqiad.wmflabs" is no submit host."
whenever I try to do much there.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Out file generator is really screwy

[Brad Jorsch (Anomie)]

On Fri, Sep 4, 2015 at 10:50 AM, Maximilian Doerr <
maximilian.do...@gmail.com> wrote:

> When using the grid to run jobs and directing the output to an out file,
> after about an hour the file becomes really screwy.  Bits and pieces get
> duplicated and pasted around, and the log ultimately gets scrambled about,
> which defeats the purpose of the log file.  This kind of makes me want to
> move off of the grid, if it's costing me my logs.
>

Are you directing the output from multiple concurrent jobs to the same
file? If so, my first guess would be that the jobs aren't atomically
appending log lines (I don't know if such atomic writes are even
supported), so the writes from different jobs wind up being interleaved
unexpectedly.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Questions regarding the Labs Terms of use

[Brad Jorsch (Anomie)]

On Fri, Mar 13, 2015 at 11:10 AM, Jeremy Baron 
wrote:

> On Mar 13, 2015 10:49 AM, "Ricordisamoa" 
> wrote:
> > What about tools and services made up of software themselves? Do they
> have to be Open Source?
>
> What are "tools and services made up of software themselves"? are there
> "tools and services" not made up of software?
>

Documentation only, maybe?


> On Mar 13, 2015 10:58 AM, "John"  wrote:
> > That's debatable. Released under a free license vs publicly available.
>
> [citation needed]
>

I suspect the debatable bit was the question "Strictly speaking, do the
Terms of use require that all code be made available to the public?",
although it's unclear due to top-posting.

Yes, it clearly states that all software has to be under an Open Source
license. But I see no requirement that the software has to be publicly
released anywhere, although it would presumably be permissible under the
required Open Source license for anyone else with access to it on Labs to
publicly redistribute it.

I don't know whether the loophole of "I write something in C, put it under
a non-copyleft license,[1] then upload only the binaries to Labs" should be
closed. Others coming later would be free to redistribute those binaries
under the license, but lacking source it would be hard to make
modifications.

 [1]: One that doesn't require source availability, e.g. BSD or MIT

-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Another labs outage - curse of the accursed hardware failure continues

[Brad Jorsch (Anomie)]

On Fri, Feb 27, 2015 at 2:27 AM, Yuvi Panda  wrote:

> ToolLabs should be back up mostly now - web tools and most bots should
> be functioning fine. Some are still down, and we're continuing to work
> on it.
>

There appears to be some fallout, I had two jobs that qstat thought were
still alive but the processes were not actually running on the exec hosts.

IRC discussion recommended qdel -f, and resubmitting the jobs once the
"zombie" job disappeared.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Database Indexing Doesn't seem to be working

[Brad Jorsch (Anomie)]

On Wed, Nov 12, 2014 at 11:06 AM, Maximilian Doerr <
maximilian.do...@gmail.com> wrote:

> My knowledge of MySQL is pathetic to be honest.  It’s definitely something
> I should familiarize myself with.  Could you explain that to me so a pickle
> could understand it? :/
>

Unfortunately, I don't speak pickle. But I'll try to clarify.


"enwiki_p.revision_userindex" is a view (along with everything else in
enwiki_p, I believe), which is basically a pretend table that's really
populated by a SELECT query (in this case, from "enwiki.revision"). You can
use "SHOW CREATE TABLE enwiki_p.revision_userindex;" to see that query if
you want. So when you select from the view, it basically rewrites your
query to incorporate that other select.

>
> +--+-+--++---++-+--+---+-+
> | id   | select_type | table| type   |
> possible_keys | key| key_len |
> ref  | rows  | Extra   |
>
> +--+-+--++---++-+--+---+-+
> |1 | SIMPLE  | revision | ref|
> PRIMARY,page_timestamp,user_timestamp | user_timestamp | 4   |
> const| 29186 | Using where; Using filesort |
> |1 | SIMPLE  | page | eq_ref |
> PRIMARY   | PRIMARY| 4   |
> enwiki.revision.rev_page | 1 | |
>
The "key" column here tells us that it is in fact using the user_timestamp
index to find the rows in the revision table. Which is good, indexing isn't
broken.

The "Extra" column tells us a few things.

   - It doesn't have "Using index", so it's having to actually fetch the
   row from the table instead of just getting what it needs from the index (in
   this case, that's because the view filters based on rev_deleted). That
   makes things slightly slower, especially if you're having to hit cold pages
   (i.e. blocks from the disk that aren't already cached in memory).
   - "Using where" tells us that it might to throw away some of the rows
   that it fetched, thanks to a WHERE clause (again, the rev_deleted check).
   - "Using filesort" means that it's going to load all the rows into
   memory (or worse, write them to a temp file) and then sort them to put them
   in ORDER BY order. This can get really slow if there are a lot of them. I
   don't know why it's deciding it needs to do this here, since the
   user_timestamp index should already be giving the rows in the asked-for
   order.

And then, of course, it's having to fetch rows from the page table as well,
which could again be hitting cold pages.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Database Indexing Doesn't seem to be working

[Brad Jorsch (Anomie)]

On Sun, Nov 9, 2014 at 1:25 PM, Maximilian Doerr  wrote:

> [1] => Array
> (
> [time] => 18.95
> [query] => SELECT rev_timestamp, 
> page_title, page_namespace FROM revision_userindex JOIN page ON page_id = 
> rev_page WHERE (`rev_user` = '14836860') AND `rev_timestamp` > 1 ORDER BY 
> rev_timestamp ASC LIMIT 0,2688354;
> [result] => succeeded
> )
>
>
The proper indexes seem to be in place on the underlying tables, based on a
"SHOW CREATE TABLE enwiki.revision" which is the backing for both
enwiki_p.revision and enwiki_p.revision_usertext.

And when I try the "SHOW EXPLAIN" trick to get an explanation, it claims
it's using an index:

+--+-+--++---++-+--+---+-+
| id   | select_type | table| type   |
possible_keys | key| key_len |
ref  | rows  | Extra   |
+--+-+--++---++-+--+---+-+
|1 | SIMPLE  | revision | ref|
PRIMARY,page_timestamp,user_timestamp | user_timestamp | 4   |
const| 29186 | Using where; Using filesort |
|1 | SIMPLE  | page | eq_ref |
PRIMARY   | PRIMARY| 4   |
enwiki.revision.rev_page | 1 | |
+--+-+--++---++-+--+---+-+

The filesort there seems odd to me, and can lead to slow queries.

Or it could just be that it's having to load 13996 rows from the revision
table and 13996 rows from the page table and the disk access is slow.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Tool labs replicas are missing the indexes?

[Brad Jorsch (Anomie)]

On Tue, Nov 11, 2014 at 8:43 PM, Maximilian Doerr <
maximilian.do...@gmail.com> wrote:

> Back to the missing index issue in regards to revision_userindex, what is
> the status of that?
>

Wrong thread.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Tool labs replicas are missing the indexes?

[Brad Jorsch (Anomie)]

On Tue, Nov 11, 2014 at 6:12 PM, Marc A. Pelletier  wrote:

> On 11/11/2014 02:49 PM, Giovanni Luca Ciampaglia wrote:
> > It's VERY unfortunate that explain does not work -- how am I supposed to
> > debug my queries then? No explain privilege => more unoptimized queries
> > => more queries will be killed => more users will be unhappy => less
> > people will use the LabsDB.
>
> It's not a question of privilege, but a change in Mysql to prevent
> explanation on views you do not have select for on the underlying table.
>  That's an upstream misfeature they do not accept as a bug.
>

Has anyone tried again with MariaDB? Maybe they'll be more sensible since
they're not beholden to MySQL's private agreement that was used as
justification when it was filed with MySQL.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Tool labs replicas are missing the indexes?

[Brad Jorsch (Anomie)]

On Tue, Nov 11, 2014 at 1:02 AM, Giovanni Luca Ciampaglia <
gciam...@indiana.edu> wrote:

> Is there a reason why the tables on the Labs replica are not indexed?
>

They are.


> Drawing a list of random titles with page_random takes more than a minute!
>

Because you're doing it wrong. "page_random > rand()" evaluates rand() *for
each row*. Since it's nowhere near constant, it can't use an index.


MariaDB [enwiki_p]> show index from page;
> Empty set (0.00 sec)
>

That's because enwiki_p.page is a view. The indexes are on enwiki.page,
which you can see the definitions of with "show create table enwiki.page".

("show index from enwiki.page" gives a permission error, likely the same
paranoia about exposing cardinality that makes a normal explain not work.)


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] speedydeletion.wikia.com

[Brad Jorsch (Anomie)]

On Mon, Nov 3, 2014 at 9:05 AM, Daniel Kinzler  wrote:

> Is "non-notable" really a reason for *speedy* deletion on enwiki?


"Non-notable" is not a reason for *speedy* deletion on enwiki. But no
"credible claim of significance or importance" is a reason for speedy
deletion for certain topic areas.[1]

Although Indian actors and Albanian football players should pass that very
low bar to avoid *speedy* deletion.


 [1]: https://en.wikipedia.org/wiki/Wikipedia:Credible_claim_of_significance
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Too big difference

[Brad Jorsch (Anomie)]

On Fri, Sep 26, 2014 at 10:15 AM, John  wrote:

> The user index table does not include anon edits


Yes it does. What it doesn't include are edits where the user field was
revision-deleted.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Too big difference

[Brad Jorsch (Anomie)]

On Fri, Sep 26, 2014 at 9:50 AM, Darko  wrote:

> MariaDB [hrwiki_p]> select count(*) from hrwiki_p.revision union select
> count(*) from hrwiki_p.revision_userindex;
> +--+
> | count(*) |
> +--+
> | 4124635 |
> | 4123974 |
>
> http://hr.wikipedia.org/wiki/Posebno:Statistika
>
> Broj uređivanja od nastanka projekta Wikipedija (Page edits since
> Wikipedia was set up)
> 4.558.675
>
> I get that some difference comes from sanitizing data, but that should be
> in range of thousands,
> not hundreds of thousands.
>

The revision table for hrwiki on the production databases matches those
numbers.

"Page edits since Wikipedia was set up" counts revisions for deleted pages,
which are stored in the 'archive' table. There are 244653 rows there for
hrwiki, both in production and in the sanitized mirror in the Labs mirrors.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] [COMPLETED] Database maintenance 2014-09-19 13:30 UTC

[Brad Jorsch (Anomie)]

On Mon, Sep 22, 2014 at 1:41 PM, Marc A. Pelletier  wrote:

> On 09/22/2014 09:59 AM, Marc A. Pelletier wrote:
> > Fixing this is my priority this morning.  Moar news coming in soon.
>
> This should now been fixed (thank you Sean!) and all databases are now
> merged into the single c3 database so that pointing to what was s3, s6,
> and s7 should lead back to the same databases.
>

Something's not quite right here:

 tools.anomiebot@tools-login:~$ sql fawiki
 ERROR 1045 (28000): Access denied for user 's51055'@'localhost' (using
password: YES)
 Make sure to ask for a db in format of _p
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] [Toolserver-l] A proposal for better tool discoverability

[Brad Jorsch (Anomie)]

On Wed, Aug 13, 2014 at 6:51 PM, Hay (Husky)  wrote:

> * You can now add multiple authors to a tool. Simply separate by
> comma, just like the keywords. I considered using arrays for
> keywords/authors, but it was a bit of a hassle on the database, so for
> now i'm sticking with commas (hopefully nobody has a name with a comma
> in it :)
>

If these names are supposed to be MediaWiki usernames, you could use pipes
to separate. If it's human names, you're probably best as just having a
freeform text field for authors unless you really need to be able to list
out the individuals.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Bigbrother is watching...

[Brad Jorsch (Anomie)]

On Wed, Jul 16, 2014 at 3:36 PM, Marc A. Pelletier  wrote:

>   jstart -N  [more options]
>

I guess I'll finally have to learn the difference between 'jstart' and
'qsub' syntax.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Cron jobs can't connect to the database

[Brad Jorsch (Anomie)]

On Fri, Jun 27, 2014 at 5:38 AM, Andre Karwath  wrote:

>  I put a small example script to http://pastebin.com/hSnhee8U
> I already tried to use an absolute path to the .cnf file, but this did not
> solve the problem.
>

I see the same error message when I use 'jlocal' in my crontab to run the
script; when using the cron job to submit a grid job to run the script it
works fine.

I suspect that the cronjob server isn't allowed to connect to the
databases. It wouldn't surprise me if this is intended, to try to avoid
people running heavy tasks on the cronjob server rather than on the grid
like they're supposed to.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Directory listing

[Brad Jorsch (Anomie)]

On Thu, May 22, 2014 at 4:09 PM, Emilio J. Rodríguez-Posada <
emi...@gmail.com> wrote:

> How can I make my tool directory browsable? In the case I don't want to
> add any index.html, etc. If I do that, the server returns a not found
> error. I'm thinking in serving some CSV files by now.
>

See the "Directory or file index" example at
https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help/NewWeb#Example_configurations?
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Tools: Mail server outage, submitting jobs from crontabs and cron jobs every minute

[Brad Jorsch (Anomie)]

On Thu, May 22, 2014 at 8:23 AM, Tim Landscheidt wrote:

> For some reason, they do that with an
> "421" status and the slightly contradictory "All messages
> from 208.80.155.162 will be permanently deferred; Retrying
> will NOT succeed."


That's dumb, I agree. Probably they want to close the connection for that
particular situation instead of waiting for the client to QUIT, and 421 is
the only status in RFC 5321 that allows the server to close the connection.
IMO a better solution would be to issue 5xx responses to every command
until the client goes away, since a well-behaved client will go away
immediately and a spammer will probably retry their spam anyway.

But SMTP really could use a "521" response code to indicate a permanent
delivery failure with connection close.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Crontab issues

[Brad Jorsch (Anomie)]

On Sun, May 18, 2014 at 8:05 AM, R Cowell  wrote:

> 1) It's trying to do my "jstop" as a grid job, which is A: a waste of grid
> resources, and B: not possible since the grid servers don't seem to have
> the grid management commands installed: -bash: /usr/bin/jstop: No such file
> or directory
>

https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Tools/Help&diff=112412&oldid=112284


> 2) The grid servers also do not apparently have 7zip installed, which is
> what my rotate script was using for
> compression: /data/project/helperbot/rotate.sh: line 7: 7z: command not
> found
>

File a 
bugrequesting
that the p7zip-full package be installed on exec nodes.

You might also submit a patch, which would look very much like
https://gerrit.wikimedia.org/r/#/c/131053/. This might speed up the process
slightly.

I also note that 7zr is available already on exec nodes. I don't know what
the difference might be between that and 7z.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Deploying files tools.wmflabs.org: this could be simpler, right?

[Brad Jorsch (Anomie)]

On Tue, Apr 15, 2014 at 8:34 AM, Pietro De Nicolao wrote:

> I use some Git repos (on GitHub) for my projects.
>

I have a similar setup, except with a private repo on Tool Labs that I just
push to. I even have some hooks set up in that private repo to
automatically do the git pull (and to restart the tool's jobs) for me.

-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] tools labs bot hammering Gerrit

[Brad Jorsch (Anomie)]

On Thu, Apr 3, 2014 at 3:47 AM, Antoine Musso  wrote:

> An example node is tools-exec-03.eqiad.wmflabs
>
> Due to the nature of tools labs, it is a bit hard to figure out which
> bot was causing the issue.  The user agent was not meaningful.
>

If nothing else, you could give Coren a list of the hammer hits and ask him
to list for each the set of jobs running on the hitting node at the time of
the hit. Then the intersection of those sets would tell us which tools to
look at.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation


On Thu, Apr 3, 2014 at 3:47 AM, Antoine Musso  wrote:

> Hello,
>
> Yesterday evening (europe time), the tools exec nodes have been
> blacklisted from Gerrit.   Some user/bot was hammering the Gerrit server
> which slowed it down.
>
> An example node is tools-exec-03.eqiad.wmflabs
>
> Due to the nature of tools labs, it is a bit hard to figure out which
> bot was causing the issue.  The user agent was not meaningful.
>
>
> If anyone had a script/bot querying Gerrit please get in touch so we can
> help fix your script!
>
> Meanwhile all tools labs nodes are blocked on Gerrit via an iptables rule.
>
> You can contact hashar, _joe_ (both Europe based) or ^d (SF based) on
> IRC.  Or reply there.
>
> cheers,
>
> --
> Antoine "hashar" Musso
>
>
> ___
> Labs-l mailing list
> Labs-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>



-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] permission/access denied issues when connecting to wmflabs

[Brad Jorsch (Anomie)]

On Fri, Mar 21, 2014 at 1:26 AM, Jeremy Baron  wrote:

> On Mar 21, 2014 1:12 AM, "Ann Samoilenko" 
> wrote:
> > torin8@tools-login:/$ cd home
> > torin8@tools-login:/home$ cd torin8
> > -bash: cd: torin8: Permission denied
>
I suppose that's to be expected:

 $ ls -ld /home/torin8
 drwx-- 3 root root 4096 Mar 17 17:30 /home/torin8

I think Coren is going to have to comment on that situation.

> Hrmmm, how did you get to / to begin with?
>
Probably sshd put her there because /home/torin8 is not accessible.

-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Image manipulation tools into Labs

[Brad Jorsch (Anomie)]

On Feb 5, 2014 3:11 AM, "Alex Brollo"  wrote:
>
> Can I install nconvert for linux into my account for personal use?

If it's available in Ubuntu, you can file a request in bugzilla for it to
be installed.
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] IMPORTANT: hotlinking of images on wikitech

[Brad Jorsch (Anomie)]

On Fri, Jan 10, 2014 at 10:26 AM, Marc A. Pelletier wrote:

> Please make sure that any content you serve to your users is done by
> your tool and not hotlinked from another non-project site.
>

Is it allowed to hotlink to project sites, such as Commons or enwiki?
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] MediaWiki to LaTeX Compiler

[Brad Jorsch (Anomie)]

On Sun, Dec 22, 2013 at 11:16 AM, Tim Landscheidt 
wrote:

> Dirk Hünniger  wrote:
>
> > The piont is that I don't want to invest much time in this
> > program anymore. If I can put it up on a server without much
> > work from my side thats Ok. But if not it is also Ok since
> > wikimedia hired someone to work on a very similar project as
> > soon as I sent them an email that my project is part of the
> > current version of ubuntu.
>
> >
> https://git.wikimedia.org/summary/mediawiki%2Fextensions%2FCollection%2FOfflineContentGenerator%2Flatex_renderer
> > First Commit 2013-11-12
>
> >
> http://lists.wikimedia.org/pipermail/wikitech-l/2013-November/073022.html
> > Sent on 2013-11-10
>
> > [...]
>
> Based on historical data, I would be *very* surprised if the
> Wikimedia Foundation could do anything in two days :-).
>

The first reference to the WMF project that I can find in a quick search
seems to be https://www.mediawiki.org/w/index.php?oldid=778814 on
2013-09-09. Announcement of the people assigned to the project (I am aware
of no new hires for the project, BTW) includes
https://www.mediawiki.org/w/index.php?oldid=816026 on 2013-11-08.

HTH.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Projects marked for death

[Brad Jorsch (Anomie)]

On Fri, Dec 13, 2013 at 9:23 PM, Maximilian Doerr  wrote:
> DO NOT DELETE XTOOLS!!

Note this is talking about
https://wikitech.wikimedia.org/wiki/Nova_Resource:Xtools, not
http://tools.wmflabs.org/xtools/.

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Redirecting from http to https with lighthttp

[Brad Jorsch (Anomie)]

On Mon, Dec 9, 2013 at 11:18 AM, Merlijn van Deen  wrote:
> _SERVER["HTTP_X_FORWARDED_PROTO"] shows up for me, and is 'http' or 'https'.

I suspect this has just been added. I see it now where I didn't an hour ago.

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] OAuth is here!

[Brad Jorsch (Anomie)]

On Fri, Nov 22, 2013 at 10:17 PM, Mahmoud Hashemi
 wrote:
> It's awesome, and works well in my development tests, but approvals take
> longer than I would hope (example).

Right now, there are only a handful of people who have the ability to
approve requests. The plan is to change the "central" wiki for OAuth
to Meta instead of mediawiki.org, and at the same time turn this
permission over to the community to manage. See bug 57336[1] for
details.

 [1]: https://bugzilla.wikimedia.org/show_bug.cgi?id=57336

> Also, any word on when the OAuth callback might be controllable in-band? I
> think I've seen the OOB-only policy some places, but not with any sites with
> which I've actually integrated (Twitter, LinkedIn).

I don't believe this is planned at all. I don't recall the details,
but the decision to allow only oob was based on a potential security
issue with allowing for in-band control of the callbacks. I'm CCing
Chris, who can fill us in on those details (I don't know if he's on
this list).


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] OAuth is here!

[Brad Jorsch (Anomie)]

On Sat, Nov 23, 2013 at 6:04 PM, Marc A. Pelletier  wrote:
> On 11/23/2013 05:18 PM, Matthew Flaschen wrote:
>> I haven't seen TUSC used for simply logging into other sites without
>> intending to take an action on a Wikimedia wiki.  It may be, somewhere,
>> though.
>
> I didn't know about that commons thing, and that is clearly OAuth.
> UTRS, however, uses TUSC just to know who you are, which is OpenID.

Other possibilities for "to know who you are":

If you're not worried about MitM (including NSA-style compromised CA
situations) or other sorts of attacks, hitting
api.php?action=query&meta=userinfo via OAuth will tell the app who the
user is. If the app doesn't have any personal or private information
or access controls based on the on-wiki identity of the user (e.g.
it's just used for "Hi $NAME" or showing/hiding "block" or "protect"
buttons based on whether you have the needed user rights), I'd think
you're probably good here.

Then there's Gerrit change 93859,[1] which would add the ability to
request what is effectively a signed version of meta=userinfo.
Something like UTRS that needs to restrict access to unblock requests
based on the on-wiki identity would need this (or OpenID).


 [1]: https://gerrit.wikimedia.org/r/#/c/93859/


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Connection Error

[Brad Jorsch (Anomie)]

According to your error message, you appear to be trying to connect to
bots-sql2.pmtpa.wmflabs with your Tool Labs database credentials.
That's not a Tool Labs database host, it's a host in the "bots"
project, so it's unlikely your credentials would work. Either use your
credentials for the "bots" project database, or connect to a Tool Labs
database host such as tools-db.pmtpa.wmflabs (note you need the
credentials from your tool for that one) or enwiki.labsdb.

On Mon, Nov 18, 2013 at 6:16 AM, Tanvi Gupta  wrote:
> Thanks Merlijn. But after correcting the hostname also, I am getting error.
> Please find the attachment for the error.
>
> Tanvi
>
>
> On Mon, Nov 18, 2013 at 4:33 PM, Merlijn van Deen 
> wrote:
>>
>> On 18 November 2013 11:52, Tanvi Gupta  wrote:
>>>
>>> Hi Merlijn,
>>>
>>> I am doing the same and getting this error. My doubt is what would be the
>>> MySql hostname.
>>>
>>> Tanvi
>>>
>>
>> Your error suggests otherwise: "Cannot open SSH Tunnel: Error connecting
>> SSH tunnel: Error connecting to SSH server: [Errno -2] Name or service not
>> known" - so it's the SSH connection that is the issue here. You probably
>> entered  'tools-lo...@wmflabs.org' as hostname -- it should be
>> 'tools-login.wmflabs.org' instead.
>>
>> Merlijn
>>
>>
>>
>> ___
>> Labs-l mailing list
>> Labs-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>
>
>
>
> --
> Regards,
> Tanvi Gupta
>
> ___
> Labs-l mailing list
> Labs-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>



-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] List of packages that are supported by tool labs

[Brad Jorsch (Anomie)]

On Wed, Nov 13, 2013 at 1:40 PM, Petr Bena  wrote:
> the reason why I started this discussion was that there is no way to
> verify if a package was already requested or not.

Sure there is: look in bugzilla for an existing bug requesting the package.

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] List of packages that are supported by tool labs

[Brad Jorsch (Anomie)]

On 11/12/13 5:34 PM, Alex Brollo wrote:
>
> Done: https://bugzilla.wikimedia.org/show_bug.cgi?id=56972
>
> You can't imagine how much I hate this kind of burocracy. Djvulibre is
> needed for djvu files management; djvu  files are absolutely needed for
> wikisource works; some busy and basic bot is working daily (IMHO) from
> Toolserver using that package, that has been simply forgotten while
> migrating from Toolserver into Labs (IMHO); I'm alerting you about this
> issue why is it not largely sufficient to go and install it (if my alert
> is right)?

"Going and installing it" correctly is more complicated than just
logging in and typing "sudo apt-get install djvulibre": it needs to be
added to that exec_environ list mentioned earlier in the email thread
so it gets installed on all the current and future exec nodes. And the
people who can take care of that from start to end (I'm not one of
them, BTW) aren't available constantly. Filing the bug means the
request is there when they have the opportunity, while just asking on
the mailing list or in IRC is liable to be missed or forgotten.


On Tue, Nov 12, 2013 at 9:43 PM, Andrew Bogott  wrote:
> We're generally pleased to accept puppet patches from volunteers.  The
> tool-labs equivalent of DIY is to submit a puppet patch that installs your
> package -- Coren and I try to be quick about reviewing such things.

I note that Coren has said in the past[1] that he has so many Gerrit
changes assigned to him for review that it's easy to miss them, while
bugs don't get missed so easily. So submitting a bug along with the
patch is a good idea.

 [1]: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-labs/20131107.txt,
at around 15:17.


On Wed, Nov 13, 2013 at 3:44 AM, Merlijn van Deen  wrote:
> It's not about puppet, it's about the response 'please jump through these
> hoops to get it done' instead of 'You're supposed to jump through these
> hoops, so I've done it for you this time', or (even better) 'It's done, and
> I have created a bug as reference'.

OTOH, the proverb "Give a man a fish and he eats for a day, teach a
man to fish and he eats for a lifetime" applies, even more so when
everyone can benefit from the mailing list post. That and I don't know
anything about djvulibre, so if any questions came up about it in the
bug I'd be unable to answer them.

> Sending a 'create a bug in this
> component' link to a mailing list is roughly as much work as just creating
> the bug directly...

Not really. Especially when I just copied the link from elsewhere.

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] List of packages that are supported by tool labs

[Brad Jorsch (Anomie)]

On Tue, Nov 12, 2013 at 5:13 PM, Alex Brollo  wrote:
> Wow! A list that anyone can read! Can please be added djvulibre package, to
> manage djvu files? Or - is there some similar package already running? I'm
> forced to migrate from Toolserver, and there djvulibre package was installed
> from many years.
>
> http://djvu.sourceforge.net/

Package requests should go in Bugzilla. You can use this link to get
started: 
https://bugzilla.wikimedia.org/enter_bug.cgi?product=Wikimedia%20Labs&component=tools&format=guided

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] List of packages that are supported by tool labs

[Brad Jorsch (Anomie)]

On Tue, Nov 12, 2013 at 9:27 AM, Marc-André Pelletier
 wrote:
> As I've said earlier in the thread, if a tool has a dependency, then it
> must be requested for and listed in exec_environ.  If you rely on
> something that isn't there*, then you are gambling.

FWIW, I threw together
https://tools.wmflabs.org/anomiebot/available-packages.php which pulls
that list and displays it in alphabetical order.

> * Except for the minimal server install, of course.  Listing fileutils
> or bash in dependencies is not useful.

A list of these would be nice. Is "perl" available in a minimal server
install? Or only "perl-base"? ;)

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Can no longer use Content-type: text/text and web server is slow

[Brad Jorsch (Anomie)]

On Mon, Nov 4, 2013 at 4:15 PM, Bryan White  wrote:
> As the speed of the web server dropped to cold tar running uphill
> levels on Friday/Saturday, would another change to the web server's
> confiig file be causing the problem?

I believe the way of the future is the new system documented at
https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help/NewWeb


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] PHP update?

[Brad Jorsch (Anomie)]

On Tue, Oct 22, 2013 at 3:50 PM, Maximilian Doerr  wrote:
> Umm...You just wrote the response it gets when it fails.

Yes, exactly. Is the data in the response from the server corrupted,
or is it getting corrupted somewhere in your code? That's a good first
step in finding out where the problem is.

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] PHP update?

[Brad Jorsch (Anomie)]

On Tue, Oct 22, 2013 at 1:35 PM, Maximilian Doerr  wrote:
> $exceptions = $site->initPage( 'User:Cyberpower678/spam-exception.js' 
> )->get_text(); mostly returns PHP Notice:  unserialize(): Error at offset 0 
> of 3xxx bytes in /data/project/cyberbot/Peachy/Includes/Wiki.php on line 596
> Where xxx is some random number.  Occasionally it returns page text.

What are the request and response when it fails?


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] PHP update?

[Brad Jorsch (Anomie)]

On Tue, Oct 22, 2013 at 8:48 AM, Maximilian Doerr  wrote:
> I’ve got this major unserialization error that seems to be happening at 
> random.

"At random" as in the same input sometimes succeeds and sometimes
fails, or "at random" as in there is no clear pattern as to which
input will succeed and which will fail but failure is deterministic
based on input?

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Cross database joins

[Brad Jorsch (Anomie)]

On Sat, Oct 19, 2013 at 6:49 AM, Maarten Dammers  wrote:
> On the Toolserver both Commons and Wikidata are on each database cluster,
> unfortunately that's not the case for labs. When can I do cross database
> joins? I would like to join Commons with Wikipedia and Wikidata with
> Wikipedia.

Last I heard there's supposed to be commonswiki_f_p and
wikidatawiki_f_p for that purpose (the "_f" stands for "foreign"), but
they seem to only exist on s1 at the moment.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Issues after maintenance

[Brad Jorsch (Anomie)]

On Fri, Oct 11, 2013 at 10:25 AM, Hedonil  wrote:
> Some files have been not rsync'd correctly. I have some files/scripts 
> backversioned to Oct 8 while others are up to date. Difficult mixture, no 
> pattern. Fixing by hand.

I haven't noticed any files that far out of date in my stuff; in my
case, it seems that changes since about October 10 at 14:55 UTC were
missing.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Discussion needed: Technically feasible, legally okay... but want tools do we want?

[Brad Jorsch (Anomie)]

On Tue, Oct 1, 2013 at 12:03 PM, Merlijn van Deen  wrote:
>
> For example, the deep user inspector's punchcard could be prevented by
> simply hiding or rounding the time of an edit (although it might also be
> necessary to rewrite revision id's, for example).
>
> It's really not a technical or legal obstacle

Err, it seems to me that rewriting the revision ids everywhere in the
database would be a major technical obstacle.

Rounding timestamps is less so, although to prevent this "punchcard"
you'd have to round them to the point where they'd be useless for many
things people *do* want to do.

Keep in mind that to prevent "evildoers" you'd really have to do this
rounding on-wiki as well: how would you like to go to
Some_page?action=history and not be able to tell when an edit was made
or reverted?


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] DNS LDAP DIT changes tomorrow

[Brad Jorsch (Anomie)]

On Thu, Sep 26, 2013 at 8:08 PM, Ryan Lane  wrote:
> If you notice any other issues, please let us know via this thread or in IRC
> on #wikimedia-labs

The DNS entries for the oauth project (w1-oauth.wikipedia.wmflabs.org,
w2-oauth.wiktionary.wmflabs.org, w3-oauth.wikipedia.wmflabs.org)
vanished entirely, no longer showing in Special:NovaAddress or
anything.

I readded them in Special:NovaAddress, so hopefully they reappear soon.


-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Access to tools-db for non-tools?

[Brad Jorsch (Anomie)]

On Sun, Sep 8, 2013 at 2:09 PM, Magnus Manske
 wrote:
> AFAIK, if your database name ends in "_p", it should be visible to everyone;
> it is private to the tool otherwise.

You could also GRANT access to other accounts, if the database name
doesn't end in _p.

But the problem here is that only tools have access to tools-db, as
far as I know; the user account doesn't. The mirrors are a different
matter.

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Accessing the databases from labs - A comparison with the toolserver

[Brad Jorsch (Anomie)]

On Jul 15, 2013 5:35 AM, "Daniel Kinzler"  wrote:
>
> That may be true if you only want to connect to one wiki's databasse. But
what
> if you want to connect to all of them, one ofter the other, or maybe just
a
> handful, as needed?

Keep a hash/dict/assoc/map of IP address to db connection, and reuse the
existing connection if you already have one for that IP?
___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l

Re: [Labs-l] Accessing the databases from labs - A comparison with the toolserver

[Brad Jorsch (Anomie)]

On Fri, Jul 12, 2013 at 11:43 AM, Marc A. Pelletier  wrote:
> On 07/12/2013 11:13 AM, Platonides wrote:
>> - How to detect if you are running in labs? (for dual tools)
>
> Possibly the very simplest way to do this would be to provide (say)
> /etc/labs on every tool labs host, testing for its presence should be a
> reliable indication.  I'll add this shortly.

WMF's puppet rules are already creating /etc/wikimedia-realm,
/etc/wikimedia-site, and /etc/wmflabs-instancename.

___
Labs-l mailing list
Labs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/labs-l