Re: [Toolserver-l] My fading out

2013-05-30 Thread Carl (CBM)
On Thu, May 30, 2013 at 10:15 AM, Nikola Smolenski wrote: > I am really sorry that you feel this way. From everything saw, as a > volunteer root you were exceedingly professional - more than most > professional roots I know. > I would like to second that sentiment. Regardless of the problems with

Re: [Toolserver-l] Cron on submit

2013-02-12 Thread Carl (CBM)
On Sun, Feb 10, 2013 at 5:48 PM, Wolfgang ten Weges wrote: > a "top" shows that the culprits are likely the same as last time : > All the CPU, and a lot of process slots (and cron slots most probably) are > currently (ab)used by /home/javadyou/pywikipedia/radeh7.py and > /home/reza/pywikipedia/ra

Re: [Toolserver-l] Cron on willow

2013-01-11 Thread Carl (CBM)
On Fri, Jan 11, 2013 at 8:33 AM, Platonides wrote: > /usr/sbin/cron does seem to be running :S It is, but something is not right. The line * * * * * /home/cbm/touch.sh at the beginning of my crontab is not being executed. The script runs fine from the command line on the same host (and a scrip

Re: [Toolserver-l] Cron on willow

2013-01-11 Thread Carl (CBM)
On Fri, Jan 11, 2013 at 8:11 AM, Platonides wrote: > Are you using Solaris crontab or cronie? /usr/bin/crontab, like always. I have checked that the same scripts work with crontab on nightshade, so it is something specific to willow. But it was working correctly on willow up until a few days ago

[Toolserver-l] Cron on willow

2013-01-11 Thread Carl (CBM)
Has anyone else noticed this? I am having problems with cron on willow, which does not seem to be executing my crontab. I tried to test it by adding a line to my crontab that should execute every minute: * * * * * /home/cbm/touch.sh That script runs correctly from the command line, but cron does

Re: [Toolserver-l] JIRA session loss

2012-11-23 Thread Carl (CBM)
On Thu, Nov 22, 2012 at 9:54 PM, Krinkle wrote: > At some point I even logged in, clicked an issue, clicked Edit (which uses > AJAX) and then the Edit screen wouldn't load due to me not being > authenticated (while I still saw my nickname on the top right). I have the same problem lately when I

Re: [Toolserver-l] Public DB status down

2012-11-18 Thread Carl (CBM)
On Sun, Nov 18, 2012 at 5:17 AM, DeltaQuad Wikipedia wrote: > It appears that at least S1 and S4 are having issues getting to the database I am seeing the same problem. I am also unable to connect to a user database on s1 from the command line: 'mysql -hsql-s1-user p_enwp10' should work, but doe

Re: [Toolserver-l] Future of the toolserver

2012-10-01 Thread Carl (CBM)
On Mon, Oct 1, 2012 at 8:12 AM, Platonides wrote: > Note that it's false that MediaWiki doesn't perform joins. > There are many of them. The statement about that was (probably) talking > about cross-db joins. Yes, that's right, I was talking about joins between user databases and the replicated d

Re: [Toolserver-l] Future of the toolserver

2012-09-30 Thread Carl (CBM)
On Sun, Sep 30, 2012 at 2:32 PM, Erik Moeller wrote: > On Sun, Sep 30, 2012 at 10:24 AM, Krinkle wrote: >> I just wanted to make sure you >> know that there are confirmed and scheduled plans for Wikimedia Labs to have >> a >> live db replication arranged between the labs cluster and the wmf prod

Re: [Toolserver-l] Reasons for not migrating to Tool Lab

2012-09-27 Thread Carl (CBM)
On Wed, Sep 26, 2012 at 2:25 PM, Ryan Lane wrote: > We currently have no plans for having the user databases on the same > servers as the replicated databases. Direct joins will not be > possible, so tools will need to be modified. This is unfortunate, and a huge step backwards from the situation

Re: [Toolserver-l] s1 replag update, suggestion, and question

2012-08-10 Thread Carl (CBM)
On Fri, Aug 10, 2012 at 8:19 AM, Russell Blau wrote: > Well, based on the overwhelming response to my last message, I guess > nobody but me cares if thyme is lagged by three or four or five > weeks I found your post very helpful for a status update of the current situation. The lag has a huge

[Toolserver-l] Tweetmeme web crawler

2012-07-23 Thread Carl (CBM)
I noticed today that "TweetmemeBot", a web crawler, was making lots of requests to my tools and ignoring the robots.txt file. The thing scrapes at a very high rate, sometimes over 30 requests per minute. Other tool maintainers (especially geohack) may want to investigate whether this is also impac

Re: [Toolserver-l] Out of memory/segfaults?

2012-05-14 Thread Carl (CBM)
On Mon, May 14, 2012 at 4:12 PM, Hersfold Wikipedia wrote: > I've gotten four error messages from my bot today, several relating to out > of memory errors or segfaults. This bot has run without error for months, > and never messes directly with memory, using the built-in Java stuff for > file hand

Re: [Toolserver-l] Dumps handling / storage / updating etc...

2011-12-12 Thread Carl (CBM)
On Sun, Dec 11, 2011 at 10:47 AM, Platonides wrote: > You seem to think that piping the output from bzip2 will hold the xml > dump uncompressed in memory until your script processes it. That's wrong. > bzip2 will begin uncompressing and writing to the pipe, when the pipe > fills, it will get block

Re: [Toolserver-l] CrontabCr

2011-11-24 Thread Carl (CBM)
On Thu, Nov 24, 2011 at 8:46 PM, John wrote: > Ive been using crontab reliably  for a long time. I havent modified my > crontab recently. At about 0230UTC 24 November crontab stopped working > correctly on willow. My scripts no longer work correctly and I am not > getting any emails from cron. Any

Re: [Toolserver-l] performance issues

2011-11-23 Thread Carl (CBM)
On Wed, Nov 23, 2011 at 6:13 PM, Platonides wrote: >> UPDATE  tmpcategories >> SET c_category = 'A-Class_Water_supply_and_sanitation_articles', >> c_ranking = '425', c_replacement = 'A-Class' >> WHERE c_project = 'Water_supply_and_sanitation' and c_rating= 'A-Class' >> and c_type = 'quality' > > I

Re: [Toolserver-l] performance issues

2011-11-23 Thread Carl (CBM)
One more DB performance report/question. I am seeing some UPDATE queries that only change one row but take much, much longer than they ought to. Is anyone else seeing this? For example, the following type of query is getting killed from time to time in the p_enwp10 database on sql-s1. The query

Re: [Toolserver-l] performance issues

2011-10-30 Thread Carl (CBM)
On Wed, Oct 26, 2011 at 9:50 AM, Ja Ga wrote: > So, is anyone else seeing really horrible mysql performance on s1? I heard > a mysql upgrade introduced regressions, but I'm seeing queries take 2x - 3x > as long, sometimes worse. > > Anoth

Re: [Toolserver-l] performance issues

2011-10-26 Thread Carl (CBM)
On Wed, Oct 26, 2011 at 9:50 AM, Ja Ga wrote: > So, is anyone else seeing really horrible mysql performance on s1? I heard > a mysql upgrade introduced regressions, but I'm seeing queries take 2x - 3x > as long, sometimes worse. > Yes. On the 25th, the following query of mine was killed after 17

[Toolserver-l] Strangely slow query

2011-10-09 Thread Carl (CBM)
I am getting a lot of emails about the following query taking too long and being killed on thyme: SELECT ns_id, ns_name FROM toolserver.namespacename where (ns_type = 'canonical' or ns_type = 'primary') and dbname = 'enwiki_p' Some of these emails claim that query ran over 3000 seconds. This p

Re: [Toolserver-l] Privacy Violation?

2011-04-01 Thread Carl (CBM)
On Fri, Apr 1, 2011 at 9:29 AM, Brett Hillebrand wrote: > Removing the Normal edit time would still allow the tool to be as useful as > it is now. A better solution would be to simply limit access to the tool and its data to pre-approved users, if this is allowed by the privacy policy. - Carl

Re: [Toolserver-l] Privacy Violation?

2011-04-01 Thread Carl (CBM)
On Fri, Apr 1, 2011 at 7:21 AM, Daniel Kinzler wrote: > The fact that you are using public data for your analysis does NOT mean that > it's compliant with the policy. As I understand it, there are three options for tools like this: * Limit the tool to trusted users, so that the analysis is not p

[Toolserver-l] Backing up a user database

2011-02-02 Thread Carl (CBM)
I realized I have to make some character set changes on one of my user databases, and I want to make a backup before I start. Of course there are server backups as well, but I view those as an absolute last resort. I was planning to use mysqldump for this. Is there a better option that I'm overlo

Re: [Toolserver-l] Removal of Perl 5.10 on Solaris systems

2010-09-13 Thread Carl (CBM)
On Mon, Sep 13, 2010 at 10:25 AM, River Tarnell wrote: > Some time ago we changed the default Perl on Solaris login servers to 5.12. > Since then, no one has reported any problems with 5.12; we will therefore be > removing Perl 5.10 later this week (that is, /usr/bin/perl5.10 and > /opt/ts/perl/5.

Re: [Toolserver-l] Changes to database access

2010-08-07 Thread Carl (CBM)
On Sat, Aug 7, 2010 at 4:41 AM, River Tarnell wrote: > * If you have any queries which could run for longer than 10 minutes when >  working correctly, add the string SLOW_OK somewhere in the query, e.g.: > >    SELECT /* SLOW_OK */ * FROM table... Does this include queries that only touch user da

Re: [Toolserver-l] CAPTCHA can be required for logins from toolserver hosts

2010-06-15 Thread Carl (CBM)
On Tue, Jun 15, 2010 at 7:29 PM, Michael Peel wrote: > Is this something that an en.wp admin can solve, or does it need developer > intervention? It needs developer intervention. - Carl ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)

[Toolserver-l] CAPTCHA can be required for logins from toolserver hosts

2010-06-15 Thread Carl (CBM)
At the moment, logins to enwiki from nightshade.toolserver.org are throwing up a CAPTCHA. This is easy to check by loading http://en.wikipedia.org/wiki/Special:UserLogin in w3m or another browser from that host. Logins from willow.toolserver.org do not require a CAPTCHA at the moment. The CAPTCHA

Re: [Toolserver-l] Linux/Solaris

2009-12-28 Thread Carl (CBM)
On Mon, Dec 28, 2009 at 3:05 PM, River Tarnell wrote: > On 28/12/2009 07:28 PM, Carl (CBM) wrote: >> 1) An easy-to-use compilation environment for C and C++. I am not >> really able to compile Perl modules on willow, despite spending quite >> a while hacking at it. >

Re: [Toolserver-l] Linux/Solaris

2009-12-28 Thread Carl (CBM)
> Am Montag 28 Dezember 2009 20:28:56 schrieb Carl (CBM): >> On the linux server, /bin/sh is bash. On Mon, Dec 28, 2009 at 3:04 PM, DaB. wrote: > that's even dangerous to suggest on the linux server. I was not suggesting anything. nightshade$ /bin/sh --version GNU bash,

Re: [Toolserver-l] Linux/Solaris

2009-12-28 Thread Carl (CBM)
On Mon, Dec 28, 2009 at 12:08 PM, River Tarnell wrote: > So: if the Linux login server were to go away tomorrow, what (if anything) > would you miss? A few things come to mind right away. 1) An easy-to-use compilation environment for C and C++. I am not really able to compile Perl modules on wil

Re: [Toolserver-l] Looking for utility to perform text search in dump

2009-07-25 Thread Carl (CBM)
On Sat, Jul 25, 2009 at 6:21 AM, Danny B. wrote: > I'm looking for any kind of tool which would take the XML dump (most probably > the > pages-meta-current.xml.bz2, at least the pages-articles.xml.bz2) and would > return > the list of page titles (or alternatively/configurably page ids) of pages