Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s[3467]

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Unfortunately this problem came back.  I will set cassia to read-only, 
and move user databases to the secondary server (hyacinth) until the 
problem is fixed.  This will cause an outage of user databases for a few 
hours.  Since we're now using separate server instances for each 
cluster, databases will be imported as describe in [0].

- river.

[0] 
http://lists.wikimedia.org/pipermail/toolserver-announce/2011-June/000440.html
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk44APQACgkQIXd7fCuc5vKRGgCeKWfizVDgfZoRjwPyKpfdTXLu
ZSwAoJ41LbAoiBzNrDtp2+o1XAEXBqQa
=SFMY
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s[3467]

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

This seems to have been some kind of controller problem.  After the 
power cycle, the server looks fine, and user databases should be 
available again.  The server might be a bit slow since the RAID array is 
rebuilding.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk41fnYACgkQIXd7fCuc5vIFfQCgrViJyviWugNVn6PEKAANDZUc
AukAn0/8kcxvm21txC8oSUnui9qCO466
=gpYs
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Hardware failure affecting s[3467]

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

One of the database servers for these clusters (cassia) has a disk 
failure of some kind.  All databases should be accessible now, but user 
databases will be missing until the problem is resolved.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk41cOoACgkQIXd7fCuc5vKekwCgnhvooTgdXbUXZ7dWotpAAX/R
vikAn0ITA4pRfpG+8TFLEnzuLnUV48K3
=kzUq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] SSH login issue

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jimmy Xu:
> It turns out that Toolserver doesn't support compression. I commented
> out "Compression yes" in my config and it started to work.

This should be fixed now.

The problem was that sshd does a chroot() in the privsep child, which 
handles communication with the client.  However, Solaris ld.so does 
lazy-loading of shared libraries by default, and sshd doesn't require 
zlib until after the chroot.  Since there is no zlib in the chroot 
directory, it failed to load:

ld.so.1: sshd: fatal: libz.so.1: open failed: No such file or directory

I have re-built sshd with lazy-loading disabled, and compression (ssh 
- -C) seems to be working now.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk4RuNIACgkQIXd7fCuc5vJ8NwCggwox2UqXzLqh+MaKvFnvgSTc
P3MAoIiKGJ8Kb0nUqkTrkNv02MZx80lR
=Vbz/
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] s3/s4/s6/s7 server split - action required for user databases

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

This mail affects you if you have user databases on s3, s4, s6 or s7.  
Other users can ignore it.

Currently, these clusters are all on the same server.   During the 
re-import, I will do an internal server split, i.e. each cluster will be 
on a different MySQL server afterwards.  We will then move user 
databases to the new servers, and do the same thing on the other server.

This presents a problem when restoring user databases, because it's not 
clear which cluster(s) the user intended each database to exist on.
That problem will be resolved as follows: User databases which contain a 
wiki name or a cluster name in the database name will be restored to 
that cluster.  All other databases will be restored to s3.

If you don't agree with this algorithm, you should list your database 
name under the appropriate cluster at [0] and it will be restored to the 
server you request.  You can list one database under multiple servers, 
if you want (but please consider disk space requirements before doing 
that).

- river.

[0]  
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk4B3HEACgkQIXd7fCuc5vJrlQCeN3lbb/GRgzihawzBWobeIttn
NAEAoLcyAN7eHNetAcpBjfH5gFEfi2BQ
=B63B
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Re-importing of several database clusters

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I'm about to re-import several database clusters from WMF: s3, s4, s6 
and s7.  This will be done on the secondary server first, so users won't 
be affected, except that queries on these clusters might be a bit slower 
for a while.

This will resolve the following TS issues:

TS-948 creating of rue.wiki
TS-926 Cassia and hyacinth are missing the ruewiki-database
TS-1013 metawiki Database seems to be corrupt

And might resolve the following issues, although they don't provide 
enough information to be sure of the cause:

TS-1024 cswikisource_p needs reimport
TS-1048 cswiktionary_p needs reimport

This reimport might take longer than usual because I will be testing a 
new way of configuring database servers (which hopefully reduce database 
corruption in the future).

Once this is done, I'll switch user databases to the secondary server 
(during which they will be offline for a couple of hours), and then 
reimport the other server.

s1, s2 and s5 will be done later.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk4AxugACgkQIXd7fCuc5vIELACdF8k2zkgsakxSo6x+jyrySy2o
KEwAnigGCyqS8td/t7gcQT97wjAwAOvV
=citf
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Python cgi safery issues

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

MZMcBride:
> There's a general expectation that the tools and scripts that you write will
> be released under a free/open source license, but I'm not sure if that's an
> actual requirement ("must") or simply an expectation ("should").

It's preferred, but not required at the moment.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk4AvMEACgkQIXd7fCuc5vK2nQCfWd4/OIjRkIXzh2B33OPeFf4c
xbEAoLtjr8PZUI9WByx/osYnvVghZkij
=AaN4
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Apache testing

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> I was looking at your encoding problem earlier; the most likely 
> possibility seemed to be that after the MySQL 5.5 upgrade, the default 
> client character set changed to UTF-8, which broke some tools.  I've 
> changed that back to latin1 to restore the old behaviour, but it 
> didn't fix your problem.

It looks like the problem is that you override --defaults-file in 
~lvova/public_html/cgi-bin/ts.  This prevents the client from reading 
/etc/my.cnf, so you don't get the correct default character set.

If you do this instead:

HOME=$(getent passwd $(id -u)|cut -f6 -d:)
export HOME

... then you don't need to use --defaults-file and your script will 
probably work correct.

(Example: <http://toolserver.org/~river/sql.cgi>.)

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3/s60ACgkQIXd7fCuc5vLYkwCglkXkbQjuKX8XeAuPaityD3kU
X5MAn0dew3FwKcwDrTXRmdKCTMd+I1Oa
=DlPv
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Apache testing

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Mashiah Davidson:
> Much better than zws. No issues with database strings encoding, compare:
 
> http://wolfsbane.toolserver.org:81/~lvova/cgi-bin/go.sh?language=ru&interface=en&listby=zns
> http://toolserver.org/~lvova/cgi-bin/go.sh?language=ru&interface=en&listby=zns

This is not an innate difference between the two, but some kind of 
configuration difference.  I was looking at your encoding problem 
earlier; the most likely possibility seemed to be that after the MySQL 
5.5 upgrade, the default client character set changed to UTF-8, which 
broke some tools.  I've changed that back to latin1 to restore the old 
behaviour, but it didn't fix your problem.

Looking at the difference in environment between the two:




... the most obvious difference is that Apache doesn't set 
LANG=en_US.UTF-8.  This doesn't seem to affect the MySQL client, though:

river@wolfsbane:~$ LANG= sql enwiki_p
...
Server characterset:latin1
Db characterset:latin1
Client characterset:latin1
Conn.  characterset:latin1

river@wolfsbane:~$ LANG=en_US.UTF-8 sql enwiki_p
...
Server characterset:latin1
Db characterset:latin1
Client characterset:latin1
Conn.  characterset:latin1

If you could add the output of SHOW VARIABLES LIKE '%character_set%'; to 
the script, and see if there's a difference between them, it would help 
track down the problem.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3/sGoACgkQIXd7fCuc5vKi1gCgtXLmDP7Q6HSNv+q5tsH6fx/9
8zYAnjE1kHL4mctgrCTvYbfsPiccaI5B
=1WLY
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Apache testing

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I've set up an Apache instance on wolfsbane for testing.  Please have a 
look at  and then test 
your tools (note the caveat about PHP, in particular).

We're not in any rush to move off ZWS, so this is mostly just 
preliminary testing.  Nonetheless, it would be useful to know about any 
problems now rather than later.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3/ahAACgkQIXd7fCuc5vI6+gCfe58t1WD4D1NYnQlACu5miPjV
WUsAnR0jWJDB2bkXFQFl2qhnPUt3iDfX
=EFB1
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Outage

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> There was an outage this morning (mostly of the web server) caused by a 
> fault on the fibre between hemlock and its storage array, which hosts 
> user-store.  I have unmounted user-store until the problem is resolved, 
> so www is now working again.

Hi,

This has been repaired and the storage is now online, so I have 
remounted user-store everywhere.

The problem seems to have been caused by a faulty HBA port on hemlock.  
Moving the fibre to the other port resolved the problem, but we will 
probably need to replace the card.

Our current purchasing plan includes hardware to provide redundant fibre 
connections to each host, to prevent problems like this in the future, 
as well as moving user-store to the HA cluster, to protect it from 
failure of a single host.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk38qmgACgkQIXd7fCuc5vKbjwCfSGc7xFz4d57Tpsya9cbvOZ1G
daUAn0QcKIpcQ9rX6iOjVoJE5H90Kva0
=mTTg
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Outage

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

There was an outage this morning (mostly of the web server) caused by a 
fault on the fibre between hemlock and its storage array, which hosts 
user-store.  I have unmounted user-store until the problem is resolved, 
so www is now working again.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk38jgYACgkQIXd7fCuc5vJyGgCgj0fRjG/YA9QHkLdBY4EVDqwu
PvYAn1Pen5ayAadqbsL9Awlbc8+sSgzk
=VtdB
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Maintenance on web servers

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Either today or tomorrow I will be reinstalling the two web servers 
(wolfsbane and ortelius).  This should not have any effect on users,
and crontabs will be backed up and restored afterwards.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk37lngACgkQIXd7fCuc5vLK+ACgp1nGvciXLPNgbn5M3R9Zl4As
b4gAn1tJaeFegx7JDO8MGFC1WuuwmYTL
=Azzf
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Maps-l] IPv6 test day on toolserver / tileserver?

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> Kai Krueger:
> > tomorrow is world IPv6 day [1]. If I have read correctly, Wikimedia is 
> > intending to participate in the test day. I was wondering if that 
> > extends to the toolserver as well and in particular to the tile server?
 
> No.

Actually, I changed my mind:

toolserver.org. 3600IN  2620:0:862:101::1:4

We'll see how it goes.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3uu4UACgkQIXd7fCuc5vL1agCgmBXeeo+OHiBLkbkCWR4lYtYt
CFMAmgIENbHGd2+BPXlV9MRjuu8vJCqt
=Mgtr
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] operating systems and web servers

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jeremy Baron:
> Also, what do you lose with FreeBSD?
 
> so far I can think of:
> * zfs dedupe

ZFS version 28, including dedup, was integrated into FreeBSD head 
(9-CURRENT) in February.  I imagine it will make it back into 8.x 
eventually, but although dedup is nice on paper, I'm not sure it's 
actually very useful. You need so much memory to cache the DDT (since 
putting it on disk kills performance) that it's usually cheaper to just 
buy more storage.

It's also unlikely that we would ever migrate the /home NFS server to 
FreeBSD (at least in the near-term future), since it lacks the HA 
clustering and volume management that Solaris has.  (This is an example 
of an area where Linux *does* have an advantage over FreeBSD.)

It's *also* unlikely that we would ever put /home on ZFS, whatever OS we 
used.  I don't mind ZFS for root, and I use it at home (on both FreeBSD 
and Solaris) without problems, but we've had some unfortunate issues 
with it at the TS that have made me a bit wary.

> * there's no longer a single "goto" company to get support from
> (guessing...) so you lose that annual cost but it may be non trivial to find
> the right hacker to find/fix a problem in an emergency. of course much of
> that support is likely to come for free (a guess)

This is true, and is something that came up the last time we discussed 
this internally.  I'm not too worried about this; the login server 
environment doesn't really tax the OS, and I can't think of many (any?) 
problems we've had there that would require vendor support.  

There are places like iXsystems  
which offer third-party support for FreeBSD.  I don't have any 
experience with that, or any idea how it compares price-wise.

> * you're still a different platform from the rest of wikimedia (basically
> the foundation) and so lose economy of scale

Not sure what economies we could gain here.  The Toolserver is 
completely separate from Wikimedia's infrastructure (except that we use 
the same colo), and both of us prefer it that way.  IOW, even if we 
used Linux, TS systems could not become just another Wikimedia server.

> * you mentioned SGE would still be available but it may be unmaintained
> (last I checked its enwp article, it said oracle was close sourcing it) just
> to keep in mind, I wouldn't stay just for SGE

SGE is dead on every platform, including Solaris.  The replacement is
either Oracle Grid Engine (which costs money), or one of the various 
forks of the open-source SGE (which should work on both FreeBSD and 
Solaris).  We're still using SGE because I want to see which (if any) 
fork gains acceptable before switching.

> things you lose with linux vs. FreeBSD:
> * zfs is gone and (assuming lvm2 + traditional RAID vs. zfs) generally thin
> storage provisioning and snapshotting is much more limited and wasteful and
> fragile/more room for error. also most storage expansions will end up w/
> transition periods that have *no* redundancy.

ZFS snapshots are nice, but OTOH ZFS can't remove storage from a pool or 
do any kind of online relayout, both of which are standard features in 
VxVM (and I assume in Linux LVM as well, although I have no experience 
of it.)

In any case we don't store data on login servers.

> * zones/jails

Linux has lxc and OpenVZ, which are both very similar to zones, as well 
as Xen or KVM for full virtualisation.

> I haven't kept up with the status in the last ~8 months but debian-kfreebsd
> may have matured enough to warrant a look. (See #debian-kbsd on oftc)

Never really saw the point of that.  If you want Linux, use Linux.  (I 
guess at the moment it's the only way for Linux users to get a useful 
ZFS?)

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3t7cAACgkQIXd7fCuc5vIBUACfcCivOSjgrnbZRSRu3W7lc+Bq
TpgAn0xAsVMmFkCnaUQTx165O6nZKyRs
=d221
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] operating systems and web servers

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Daniel Kinzler:
> why FreeBSD and not Linux?

Why Linux and not FreeBSD, or Solaris, or OpenServer, or MP-RAS, or ...?  
I've asked this many times in the past (usually when someone says "I 
want Linux") and never had a real answer that I can remember.[0]

The way I see it, choice of operating system has very little impact on 
users because users don't, generally, interact with the operating system.  
There are a few specific cases which make a difference (I can think of 
cron, 'ps' and top/prstat), but apart from that, the software users 
actually use is the stuff in /opt/ts, which is independent of operating 
system.[1]

For servers other than login servers (like databases) there are 
additional considerations, since those make a lot more use of OS 
features, like storage management.  I've previously considered and 
rejected FreeBSD for use as a database server (in favour of staying with 
Solaris + VxVM).  

To me, FreeBSD seems to be an ideal candidate for a platform to layer 
/opt/ts on top of: it provides a base operating system which is 
reliable, has a good feature set in areas which only the OS can provide 
(e.g. DTrace, ZFS, auditing), and it's simple to install and maintain.

- river.

[0] I know some people are concerned about use of proprietary software 
on the Toolserver, but that's a separate issue, and in any case FreeBSD 
and Linux are equally open source.

[1] This assumes that all the software we provide works on FreeBSD, of 
course.  I haven't verified this, because we provide ~650 packages and 
it would take many hours, so there's no point doing it unless we 
actually decide to change OS.  However, I don't foresee any problems.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3t5ikACgkQIXd7fCuc5vLokACfUbRk0fIc6yU6tdXaS+oTr26/
0G8AoMGuJ149lizDh9KMO426taRJVDr8
=Hom2
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] operating systems and web servers

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

So, this is not an announcement of any intent to change anything, I just 
want to get an idea of how people feel about two things we could, 
perhaps, change in the future:

1. ZWS to Apache on the web server
2. Solaris to FreeBSD on login servers

#2 depends on #1, so it seems sensible to discuss both together.

I don't have any strong opinion about either of these myself, but I 
would like to hear what users think.

ZWS to Apache:

I know it annoyed people when we moved from Apache to ZWS initially, 
because rewrite rules had to be redone, some .htaccess stuff stopped 
working, etc.  At the time we were using mod_suphp for per-user (suexec) 
PHP scripts, and it was extremely inefficient; the system spent most of 
its time doing nothing.  ZWS allowed us to fix the problem cheaply (no 
new hardware required), and I think for most users it worked just the 
same.

Since then, two things have changed: firstly, Apache with mod_fcgid now 
has decent FastCGI support, and with only a little work could be made to 
support suexec PHP as well.  Secondly, ZWS is now in maintenance, and 
won't see any further development (so it might be better to switch now, 
rather than wait until one month before ZWS support ends entirely and be 
forced to switch).

With that in mind, it makes sense to consider moving back to Apache.  
The main downside is that rewrite rules would have to be converted back 
to Apache format (mod_rewrite).  OTOH, .htaccess features missing from 
ZWS would be available again (I don't know if anyone actually needs 
this, but I believe at least a few users have complained about missing 
features.)

Solaris to FreeBSD:

Of the two changes, I think this one would actually be the less 
disruptive.  For users, nearly everything would stay the same: we 
already provide the GNU userland ('ls', etc) by default (and would 
continue to do so) and the third-party software in /opt/ts would be 
identical, as would cronie, SGE, Perl/Python/..., etc.  

Software-wise, since nothing would really change, I don't see any 
particular advantages for users.  Disadvantages: 'ps -eaf' would stop 
working ;-) and anyone with locally-compiled software (C/C++, or XS Perl 
modules, etc.) would need to recompile them.

For us (admins), the main advantage is reduced maintenance overhead: 
FreeBSD releases a new minor version about once a year, and supports 
each for 2 years; each release branch only gets very infrequent updates 
for security or errata.  In comparison, there is a new Solaris update 
every 6 months, and during yesterday's maintenance we installed 358 (!) 
separate patches.  Oracle doesn't provide a security-updates-only 
release, and it's difficult to mix-and-match patches (e.g. to only get 
security patches).  

This doesn't directly affect users, but fewer OS changes should lead to 
less lengthy / disruptive maintenance and less frequent reboots.  OTOH, 
I don't know if this has a noticeable impact on users at the moment, and 
the previous maintenance was the first for ~170 days.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3thyIACgkQIXd7fCuc5vJjsgCfYfbW28r+eW3Lr6L1T+BrzGV/
v4YAn3lkZ7+YfG8ajKBQRx7DBQRKhfnS
=lcFN
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] General maintenance notice: Monday, June 6th

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

This maintenance was mostly successful, except:

* An unexpected problem upgrading software on the NFS server caused an 
  NFS (/home) outage of around 20 minutes.

* The Python module "pyexiv2" is currently unavailable.  This will be 
  fixed tomorrow (7th).  (Sorry.)

* Login servers will be rebooted again tomorrow for some final cleanup.

* FishEye upgrade was postponed until tomorrow.

In case of any other problems, please file an issue in JIRA.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3sxuoACgkQIXd7fCuc5vJqZACfe3ZyFcPV+GZZQUz0s62myOby
2SUAni0qQBIhWau0TFK8jAlYjQr+UuCA
=1ZYm
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: Monday, June 6th

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Shubinator:
> On Tue, May 31, 2011 at 2:41 PM, River Tarnell
> wrote:
> > Start time: Monday, 6th June, 0800h UTC (or possibly earlier)
> > http://time.tcx.org.uk/utc/2011-06-06/00:00

> Your text says the maintenance will start at 08:00 UTC, but the link implies
> 00:00 UTC. Which is it?

0800h is the correct time.  <http://time.tcx.org.uk/utc/2011-06-06/08:00>

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3l6GQACgkQIXd7fCuc5vKkdACgj3m7NEJ7hQzq2Kj2WwVg5PTv
9KEAoJKUEZub4RqAbcYzH4CCy1fkqXWG
=jQtP
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] General maintenance notice: Monday, June 6th

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

On the morning (UTC) of June 6th we will performance general maintenance[0] 
on all servers.  Services will be affected as follows:

 Service   | Expected impact
 --+--
 Entire platform   | As described in maintenance schedule[0]
 FishEye   | Unavailable for < 10 minutes

[0] https://wiki.toolserver.org/view/Maintenance_schedule

Start time: Monday, 6th June, 0800h UTC (or possibly earlier)
http://time.tcx.org.uk/utc/2011-06-06/00:00

End time: Monday, 6th June, 1200h UTC (estimated)
http://time.tcx.org.uk/utc/2011-06-06/12:00

Details:

We will install current operating system patches on all servers, which will
require a reboot of each system.

 --

We will enable IPv6 on the NFS server, which might make /home 
unavailable for a short period even if hosts are up.

 --

FishEye will be upgraded to 2.5.5.

 --

We will perform general software upgrades for ts-specs (/opt/ts).  A list of 
software to be upgraded can be found at:

  

Some software may be unavailable or function incorrectly during the upgrade 
process, which we estimate will take under 30 minutes.

Note: Mono will not be upgraded due to a build failure which was not 
fixed in time for the maintenance.

ts-specs (/opt/ts software) changes
- ---

We now build software with GCC stack-smashing protection (-fstack-protector) by 
default, and several packages have been rebuilt to benefit from this.  This 
should not cause any user-noticable changes.  

Some notable changes are detailed below:

webp
- 

The "webpconv" binary is no longer provided; instead, use cwebp and dwebp.

OpenSSL
- ---

We will install a set of root CA certificates for OpenSSL, which will enable
SSL connections (e.g. from cURL or wget) to work by default, as long as the
certificate is valid, rather than requiring the user to provide a certificate
or disable checking.

The set of installed certificates will be the current Mozilla root certificate
set (from Firefox) and the Toolserver CA certificate from 
https://fingerprints.toolserver.org.

Python 3
- 

The default version of Python 3 (/usr/bin/python3) will change to 3.2.  Python
3.1 will be removed during the following maintenance.

MySQL
- -

The MySQL client will be upgraded to 5.5.12, and will move from 
/opt/ts/mysql/5.1/bin to /opt/ts/bin.  If you currently call "mysql" without a 
path, you do not need to change anything.  If you use 
"/opt/ts/mysql/5.1/bin/mysql", you should change to "/opt/ts/bin/mysql" (or 
preferably remove the path and rely on $PATH).  The old (5.1) client will still 
be available for now.

The MySQL client library will also move to /opt/ts/lib.  The old client library 
will still be available, but if you have any compiled software which links 
against MySQL, you should re-compile it with the client library in /opt/ts/lib.

libpng
- --

libpng has been upgraded from 1.4 to 1.5.  A 1.4 runtime library is provided 
for compatibility, but if you have any software that links against libpng, you 
should recompile it with 1.5.  The following warning (from the libpng 
documentation) applies to this upgrade:

The libpng 1.5.x series continues the evolution of the libpng API, 
finally hiding the contents of the venerable and hoary png_struct and
png_info data structures inside private (i.e., non-installed) header
files. Instead of direct struct-access, applications should be using 
the various png_get_xxx() and png_set_xxx() accessor functions, which
have existed for almost as long as libpng itself. (Apps that compiled 
against libpng 1.4 without warnings about deprecated features should 
happily compile against 1.5, too.)

GCC
- ---

GCC has been upgraded to 4.6.0.  This should be backwards compatible, so there
is no need to recompile software.  There are two relevant changes for C++ users:

* If you define _XOPEN_SOURCE, you need to use -D_XOPEN_SOURCE=600.  
  -D_XOPEN_SOURCE=500 will not work.

* GCC 4.6 will no longer accept a const object without a ctor, i.e.:

struct S { };
const S o;

  The fix is to either add an empty constructor, or explicitly 
default-initialise
  the object:

const S o = S();

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3lYHIACgkQIXd7fCuc5vKCpgCfTKXkByYIjD8f7sFhSRk+kMSl
BksAoI86/sVpyhFt6YoFpYjI+OUS+OQj
=+Pc2
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] SSL

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

It is now possible to access toolserver.org via SSL, e.g. 
.  This is otherwise identical to normal 
HTTP access.

Unfortunately, because of how this is implemented, it's not possible to 
require that SSL be used for a particular page, or for a CGI script to 
check if SSL is in use.  If anyone has a need for this, it might be 
possible to implement  or similar.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3lXzYACgkQIXd7fCuc5vKg7ACgiwaaPDKhhabddeQsQI0la0OJ
rFkAn3/jyhY6XyupfT/FozxbpfK6MHiz
=9+QL
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Change to toolserver.org HTTP

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I have switched toolserver.org HTTP to use a Squid reverse proxy for 
load-balancing instead of Solaris Cluster.  In case of problems (like 
odd HTTP errors), please file an issue in JIRA.

NB: This is unrelated to the previous IPv6 changes.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3kKkUACgkQIXd7fCuc5vK+JACglpYP1/yEV0k66fByrDT+Ztom
x7QAmwXkPp9zXkhXRgIs6u4g5+ob4INh
=vX+x
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] IPv6

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

During the next maintenance window I plan to start the (somewhat 
overdue) deployment of IPv6 at the Toolserver.  In preparation for this, 
I will be adding IPv6 addresses to all hosts today.  

I don't expect this to cause any problems for users, except that the IP 
address for willow (which already has one) will change.  I will leave 
the old IP in place until everything has migrated, but the new IP will 
be used for outgoing connections.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3h5w4ACgkQIXd7fCuc5vIc4QCggvuH0CpZ1scPjb82g8xjxcmX
IkoAoK8xKuYW2WO8Zqv4T8qUbamZPAvQ
=c2zi
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Suggestions page

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I have added a page on the wiki to suggest new features or changes to 
the Toolserver: .  If you 
have a suggestion (even if you think it's not feasible), you can add it 
to that page we will consider it.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3f+5oACgkQIXd7fCuc5vK55wCgn6AYfpneISzv50AXWradPBFm
mpUAnjC0680tZVpRBJvR3waEnUa2nqlU
=l1UE
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Changes to mail sending

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

At some point in the future we plan to make some changes to how mail 
sending on the Toolserver works.  Unless you have tools which send mail 
(not including mail from cron or SGE), this change does not affect you.

Specifically, we will no longer allow users to inject mail into the 
Toolserver mail system via SMTP, unless the mail is to a Toolserver 
address; in other words, internal hosts will be treated identically to 
any Internet host.  This means you cannot send mail via SMTP to
mail.toolserver.org or localhost.

It will also be forbidden to send mail directly to Internet hosts, 
but this will not be enforced by technical means.

If you send mail using /usr/lib/sendmail (or a comparable mechanism, 
such as /usr/bin/mail or Mutt), you do not need to do anything.  (This 
includes PHP's mail() function.)

If you send mail via SMTP, you should stop doing that, and instead use 
/usr/lib/sendmail.  This generally means invoking 
"/usr/lib/sendmail -oi -bm -- " and sending the mail body to 
it on stdin (including headers).  Remember to escape any shell 
metacharacters in the address, if applicable.

It should generally be trivial to convert anything that uses SMTP to use 
sendmail instead, and you should do this now rather than waiting.  If 
you think this is not possible for some reason, you should let us know 
sooner rather than later.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3f5k8ACgkQIXd7fCuc5vLKwQCgnb388KrYECUXroxuogkART3p
VMgAn1vZGwNyr7POKIKqP+DM3gxYpcN3
=7Dq0
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Change of SSL certificate for svn.toolserver.org

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

The SSL certificate for https://svn.toolserver.org has changed.  The new 
fingerprint is available on  and 
also below:

SHA-1: E8:62:65:9A:89:CA:1C:0D:8B:97:80:93:F3:CA:04:F7:5F:B8:A8:D5
SHA-256: 
4D:45:9B:60:E1:82:F6:57:4B:D0:EB:66:1C:22:25:21:95:24:2E:5A:7C:A6:C1:BC:B7:6D:FC:F4:6C:AF:84:E4

Your SVN client may require you to (re-)accept this key before you can 
access the repository.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3eYHMACgkQIXd7fCuc5vKW4ACdGqJNwbqxWfe8h828VNSkJjb/
pxAAoJ3hF/V8vC4G/wcsf/nBBVAVqOKd
=ebwa
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Change to misc sites web server

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

The front-end web server on amaranth (which serves JIRA, MediaWiki, 
FishEye and WordPress) has been changed from Sun Java System Web Server 
to Apache.  This should not cause any noticeable changes, but please 
report any problems to JIRA or ts-adm...@toolserver.org.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3T3OEACgkQIXd7fCuc5vJr8wCfYRN6DaltlbGWX+qfDv/R4oLe
mSEAniY5Yfoj2VvWV1wsE+1Db8oWQb1/
=D6af
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Replag

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

John:
> Ive seen the primary sql servers for 1,2/5 lagged about 12 hours for the
> last day.

One of the s1 servers (thyme) was lagging; probably because a user 
allocated a very large MEMORY table and caused the server to swap.  I 
restarted and server (and fixed the limit on MEMORY table size), and it 
seems to be doing better now.

I don't see any particularly serious lag for s2:


Unfortunately we don't have replication lag graphs for s5 at the moment.  
If there was lag there, it's probably because the s2/s5 server (daphne) 
is somewhat overloaded, since there's only one of them, while other 
clusters have two servers.

> While the fast servers are current. 

There are no fast servers anymore; sql-sX-rr and sql-sX-fast both point 
to both servers for each cluster (except s2/s5).

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3Q4egACgkQIXd7fCuc5vLq6ACfdx+WpdbVlVaE47qsFoV4grBV
ShwAoKRMdQmVtOiD7EpVhs/YKHZc8et1
=8cOZ
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] s1 servers changed

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> Due to high replication lag on thyme (s1) I've switched 'sql-s1' to 
> rosemary.  If you have scripts which access user databases on s1 and 
> haven't been updated to use sql-s1-user instead, you will find your 
> databases are missing.  The fix is to connect to sql-s1-user instead of 
> sql-s1.

This has now been reverted.

However, anyone who still connects to 'sql-s1' assuming it holds user 
databases should still change to using sql-s1-user.  It's likely that we 
will eventually make this change permanent, to improve load distribution 
across the two servers (since many people still use sql-s1 instead of 
sql-s1-rr).

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3PTD8ACgkQIXd7fCuc5vJk5gCZAf1/R+xcDEzzYyykntIv3h1Z
ElYAoJRbnx8dd6Se6WKS/1KzWda11cWh
=lFTs
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Schema-only database backups

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

It is now possible to mark a user database to be backed up without its 
data, i.e. only the schema is dumped (mysqldump -d).  To do this, 
include the string "_transient" in the database name:

u_jsmith_transient
u_jsmith_my_transient_data_p

If you have databases whose data doesn't need to be backed up, it would 
be very helpful if you could rename them to include _transient in the 
name, to reduce the load / disk space requirements of the nightly backup 
job.

Unfortunately MySQL doesn't provide a way to rename a database, but you 
can copy the database to a new name like this:

$ mysqladmin create u_jsmith_transient
$ mysqldump --opt u_jsmith | mysql u_jsmith_transient

The old database should then be dropped.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3OGlQACgkQIXd7fCuc5vKdsQCaAwWY3LJ/H4+JcvB0x7VHTyzS
VSMAnjVCQBvlfOVG/2DmLtUz0c9EUe5Y
=sB8i
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] s1 servers changed

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Due to high replication lag on thyme (s1) I've switched 'sql-s1' to 
rosemary.  If you have scripts which access user databases on s1 and 
haven't been updated to use sql-s1-user instead, you will find your 
databases are missing.  The fix is to connect to sql-s1-user instead of 
sql-s1.

Scripts which do not use user databases will not be affected and should 
not be changed.

This change will be reverted once thyme has caught up.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk3OC2QACgkQIXd7fCuc5vIz7ACdENxlPUZHXengxHE/ZcYp4hkt
g5oAoLTAXH4VJWJVZCla5sAhQ25sniRh
=9Jc7
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Perl 5.14 now available

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Perl 5.14.0-RC1 is now installed as /usr/bin/perl5.14.  5.14 will 
eventually become the default version of Perl; you may want to test your 
Perl programs with it before then.

This is a pre-release version of Perl, so it may contain more bugs than 
usual.  Please report any issues in JIRA.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk28ggkACgkQIXd7fCuc5vI9agCgws7XiUDPVlSsaOeDzae0jxcV
eTIAnjChG53PvDAwBo93ux4FhS3fmHxU
=3qSA
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] Maintenance: JIRA, FishEye: Saturday 30th April

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> Tonight (UTC) we will upgrade JIRA and FishEye to the latest version.  
> This will involve around 15 minutes downtime for each service.

This is now finished.  Downtime for JIRA was longer than expected (about 
30 minutes) but within the maintenance window.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk27YJQACgkQIXd7fCuc5vIhPACeMA4ZquPHssQOdUc9vFUQFkDb
DjwAn0TU8hU5cVGNM8s1SWJsHgj5VvNF
=wC4e
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Maintenance: JIRA, FishEye: Saturday 30th April

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Tonight (UTC) we will upgrade JIRA and FishEye to the latest version.  
This will involve around 15 minutes downtime for each service.

Start time: Saturday, 30 April 2011, h UTC
http://time.tcx.org.uk/utc/2011-04-30/00:00

End time: Saturday, 30 April 2011, 0100h UTC
http://time.tcx.org.uk/utc/2011-04-30/01:00

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk2616EACgkQIXd7fCuc5vICXQCfWdhs8DjcqZJ1a2exNODKdB5j
BnIAnjopZFef9aJyWzOdqccRrecRynZ0
=h0Vs
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] PGP support for ts-admins address

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

The ts-admins contact address is now PGP-enabled.  This means all mail 
coming this from address will be signed, and if you send encrypted 
email, you will receive encrypted email in reply.  Notifications to 
admins are also encrypted, and the ticket system's web interface is 
accessed via SSL, so your mail should never be sent over the Internet in 
plaintext.  

However, mail is stored unencrypted in the database, so this is not 
secure if the ticket system itself is compromised.

For this to work, your key needs to be available from the PGP keyserver 
network (the ticket system uses wwwkeys.uk.pgp.net).

The key ID for ts-admins is:

pub   4096R/E04DCDD7 2011-04-13 [expires: 2012-04-12]
  Key fingerprint = 5A5E 8634 DD8E DB05 F36E  3969 7035 1043 E04D CDD7
uid  Wikimedia Toolserver ticket system 

uid  Wikimedia Toolserver ticket system 


... which is available from wwwkeys, and is also listed at
.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk2l9k8ACgkQIXd7fCuc5vIK3ACfbtDHqppbthS4PYQgMUeg/QSK
GdoAoL2dhZOIgP6W1jWZ9JogIh/F/Fi1
=BIvL
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Alternative C function to vasprintf() on Toolserver

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Andrew Dunbar:
> Hmm currently I'm using _vscprintf / vsprintf on Windows/MSVC inside
> #ifdef _WIN32
> and vasprintf Ubuntu/gcc inside #else
 
> But my C-fu is to puny to figure out what #ifdefs to use to detect
> what combination of C compiler and libc I'm compiling under.

> Apparently it's important with snprintf since some implementations
> return "an unspecified return value less than 1" with size=0.
 
That's true, but no current Unix implementation should do that, since 
the return value of snprintf() is standardised in the current POSIX 
standard (IEEE 1003.1-2004) and in the C99 standard (ISO/IEC 9899:1999).

The following code should handle Windows and both old and new Unix, 
although it's completely untested and comes with no warranty:

/* Compile with "-D_XOPEN_SOURCE=600 -std=c99" */
int
my_vasprintf(char **res, char const *fmt, va_list args)
{
int  sz, r;
#ifdef _WIN32
sz = _vscprintf(fmt, args);
#else
sz = snprintf(NULL, 0, fmt, args);
#endif

#if defined(_WIN32) || (defined(__STDC__) && __STDC__ >= 199901L) \
|| (defined(_XOPEN_VERSION) && (_XOPEN_VERSION >= 600))
if (sz < 0)
return sz;
if (sz >= 0) {
#else
if (sz >= 1) {
#endif
if ((*res = malloc(sz + 1)) == NULL)
return -1;

if ((sz = sprintf(*res, fmt, args)) < 0) {
free(*res);
*res = NULL;
}

return sz;
}

#define MAXLN 65535
*res = NULL;
for (sz = 128; sz <= MAXLN; sz *= 2) {
if ((*res = realloc(*res, sz)) == NULL)
return -1;
r = vsnprintf(*res, sz, fmt, args);
if (r > 0 && r < sz)
return r;
}

errno = ENOMEM;

if (*res) {
free(*res);
*res = NULL;
}

return -1;
}

Obviously, this will be slower than the modern implementation, and you 
may want to tune the initial buffer size for the expected string length.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk2i0QcACgkQIXd7fCuc5vI1ewCfTdlavg0rr56l7JzwAe5Oea9x
55sAnA4MIosJLfOnD/O3z/nkq0yqD5CE
=AGMe
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Alternative C function to vasprintf() on Toolserver

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Andrew Dunbar:
> I've got a little program to index dump files that supports Windows
> and Linux but it doesn't compile on the Toolserver with either cc or
> gcc due to the lack of the function vasprintf(). It's a GNU extension
> so I'm surprised it didn't work even with gcc.
 
> Why doesn't the Toolserver gcc have it, and does anybody know of a workaround?

*printf() is not part of the compiler, but the C library (libc); GNU 
glibc (used on Linux) provides it, while Solaris doesn't.  There's been 
some discussion about it, but there are several incompatible asprintf() 
interfaces around and it's not clear which one should be implemented.

Someone else already provided a version using snprintf, but you could 
also use g_vasprintf() from glib, or the version from gettext's 
libintl.h (which means you need to include  and link against 
- -lasprintf, both of which live in /opt/ts/gettext).  I personally find 
the snprintf solution the best and this is what I use in my own code.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk2gaJ4ACgkQIXd7fCuc5vKziwCfTbDgK0nX32jK8iJcLWl278BK
HNwAnRFW8/BhMUKS7zr6RRBcPlqCeWfj
=31oq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to mail forwarding

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

José Emilio Mori Recio:
> Regarding this [old] subject, and saying hello to everyone in my first
> mail here, it seems that $HOME/.forward (containing $h...@toolserver.org)
> should be no longer created for new accounts. After testing in my
> few-days-old account, external mail sent to j...@toolserver.org was going
> apparently to nowhere, until I deleted .forward and then began to arrive
> to my LDAP address, as expected...

You're right, this file should not be created any more, and doing so 
will cause mail to be lost (actually, bounced, I hope).  I've fixed 
mkuser to no longer create it and removed the bad file from recently 
created accounts.

Thanks for reporting this -- no need to file an issue in JIRA.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)

iEYEARECAAYFAk2OMEAACgkQIXd7fCuc5vJn8wCfVNDG68inKHUTWePesX3tQ9KT
OV4An0at+McdSuQGrmm6zsWWjNd5a09B
=fBHO
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] sql-s1 replag

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Due to the current high replag on thyme, I've switched sql-s1 to 
rosemary.  If your tools require a user database and you haven't updated 
them to use sql-s1-user instead, you will find your databases are 
missing.  Once the replag is down to a reasonable level I'll switch it 
back.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk13HAsACgkQIXd7fCuc5vIvdQCffgfjFv9OKmvpoWNY/0+LiTqy
IeUAniODzwDbdQG7jcf3Jp/3awc40gLR
=MBKi
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Schema changes, now

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

The schema changes are now complete; unfortunately it took slightly 
longer than expected.  During the maintenance I switched the primary 
server for s[3467] from hyacinth to cassia.  hyacinth will be 
unavailable until tomorrow, so for now these clusters only have a single 
server.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk12KJ4ACgkQIXd7fCuc5vKoKACggmBrsgltzTx0qWpylZ/C1DlE
A3IAoIx8ZuFRp0EFax7VUmMVxi6uE+/J
=zQ0I
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] New SGE execution host

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I've added yarrow as a login server and SGE execution host for all.q and 
longrun.  All SGE jobs are now eligible to run there; no action is 
required from users.

This server is for SGE only, tools must not be started there manually.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk11fv4ACgkQIXd7fCuc5vK24wCgjGrOQSwxLdndb3FeVbN/Dr4l
FbQAn1GMC00MN8vDklXQqFBgDVv1mS7J
=yQue
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Schema changes, now

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

We're currently applying schema changes from WMF for a MediaWiki 
upgrade.  To reduce replication lag, we will first apply them to an 
inactive server for each cluster, then switch users to that server and 
apply them on the other one.  The first part has started now, and we 
will be switching in an hour or two.

During the second part, user databases will be unavailable until we 
finish the changes and switch back.  This may take several hours to 
finish.

The exception is s2/s5, which only has one server; these clusters will 
be lagged during the change.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk11bNYACgkQIXd7fCuc5vIIowCgqKiJvPxtr+B19TRDrbJTy/70
fHQAn3CW69mS7tQPWrRxdMTKYnuFM5St
=DpI/
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <4d74c74b.4010...@mathgen.ch>,
Frederic Schutz   wrote:
> > * Dumps will go in /store/dumps//
> > * Page view stats will go in /store/stats/
> > * Everything else will go in /store/misc/
> This looks good to me. I am ready to populate the stats system as soon 
> as it is ready to be used. Have you already decided the size ? It could 
> do with 3 Tb...

Can you estimate the rate of growth for the stats?  That would help in 
planning the size.

The change probably won't happen immediately, since our current (free) 
VxVM license only allows 4 volumes.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk11JxoACgkQIXd7fCuc5vI6sQCeODn6bahLCSI0uyzL19Kgyw+j
E4UAoKryjpQeBZ70jgYmmjFjgZA/083g
=lT6X
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Maintenance tonight: mail service

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> Tonight I will implement the mail forwarding changes described here: 
> <http://lists.wikimedia.org/pipermail/toolserver-announce/2011-February/000403.html>.

This is now done.  Please check everything works as you expect, and file 
any problems in JIRA.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk10MbEACgkQIXd7fCuc5vIS7QCfVmKcARn/NX1yeePVaHWqXtWz
sMMAnjQagfEffogHL0KlK1ui3UtspCCD
=BjHE
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [SECURITY] SSH / Password login

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <4d73dec0.9030...@gmail.com>,
Platonides   wrote:
> Wouldn't such login have been logged? Seems easy to find out if any
> account was accessed this way.

No, because the problem was introduced in December and we don't keep 3 
months' worth of old logs around.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1z3lIACgkQIXd7fCuc5vIokQCeMOScxP+wvLasspBfas+HQ5yA
09UAn1eyDgwbhy03C6jM3w5/TW/MRmPF
=wU1u
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Maintenance tonight: mail service

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Tonight I will implement the mail forwarding changes described here: 
.

There should be no interruption in service, except that mail to 
u...@toolserver.org might be delayed during the maintenance.

Start time: Monday, 7th March h UTC
http://time.tcx.org.uk/utc/2011-03-07/00:00

End time: Monday, 7th Match 0100h UTC
http://time.tcx.org.uk/utc/2011-03-07/01:00

Sorry for the short notice.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1z3dgACgkQIXd7fCuc5vJtSwCgiZO5Mu3pMIOJrhcuYUekOHvw
awQAni555uAR0bEORIUV869VqSlS1hT/
=kRiq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [SECURITY] SSH / Password login

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article ,
Marcin Cieslak   wrote:
> > If you have no LDAP password set, (...)
 
> How do I find out if I have one? I don't remember I ever set this,
> or may be it was long ago.

Run setpass:

willow% setpass
setpass: password already set
setpass: use passwd(1) to change your password

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1zwhoACgkQIXd7fCuc5vImCQCguZ5c4S6yuVxuzSnK04gl64/G
Yn0An389qvaqjjwKd+2vpOw2a3o5cgJ4
=GoHp
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] [SECURITY] SSH / Password login

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

During the maintenance on December 6th, 2010 I switched the Toolserver 
SSH server from Sun SSH to OpenSSH.  A difference in how OpenSSH uses 
PAM to authenticate users meant that after the change, users were able 
to log via SSH using their LDAP password, without using an SSH key.  
This error has now been fixed.

If you have no LDAP password set, or if you have a strong password[0], 
then this should not have affected you.  However, if you had a weak or 
easily guessable password set, or if your LDAP password could have been 
compromised (e.g. if you wrote it down in plain text somewhere) then 
it's possible someone could have used it to gain access to your account.  

In that case, I suggest you immediately change your password (via 
'passwd'), then review your home directory to ensure no unauthorised 
changes have been made (e.g. new SSH keys added, or shell rc files 
changed).  If you have sensitive data such as SSH or PGP keys on the 
Toolserver, you may wish to revoke them and issue new ones.  (However, 
storing that kind of data on the Toolserver is probably a bad idea in 
any case.)

I'm very sorry for the inconvenience this issue might cause to users, 
and I will be reviewing our authentication configuration to reduce the 
chance of something like this happening in the future.

- river.

[0] Which is somewhat enforced by the LDAP password policy, but it's 
still possible to set a weak password if you try hard enough.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1zvOgACgkQIXd7fCuc5vJpowCeMoLig31BAHnStWakKgeU/ZOr
pCYAoKMEF/6+yzzKGQNVYxXqJuhM2f63
=ykB1
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> I'm currently copying user-store to a new disk array; this should be 
> finished in a couple of days.  I hope the new array will be faster, as 
> well as much larger.  There will be a separate maintenance notice for 
> the actual switch later.
 
This is done now; in the end I did the switch without a maintenance.  
Some data is still copying (should be done in about 4 hours), so it'll 
be slower until then.

> Once the move is done, in additional to having more storage now (about 
> 14TB), it will be much easier to add additional storage in the future.  
> However, I don't think having a single 14+ TB filesystem is a good idea 
> (the initial volume will be 5TB), so I'm considering new ways to split 
> up the storage.

Well, no one commented on this, so I propose the following:

* Dumps will go in /store/dumps//
* Page view stats will go in /store/stats/
* Everything else will go in /store/misc/

Symlinks from the old locations will prevent breaking tools which access 
data in its current location.  Projects which need a lot of disk space 
(say, 100GB+) will be able to request a volume at /store// for 
their files.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1zCtwACgkQIXd7fCuc5vJwIACgmNFo5NYc0PENM0V7UQp8npLo
o/EAoJP0JWAZm7leh7xdFB0QA2Qqn2eQ
=HIOq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store maintenance, tonight

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Later tonight (around 12AM UTC[0]) I will restart the NFS server on 
hemlock, which serves user-store.  This will cause an interruption to 
service that should last for under 1 minute.

- river.

[0] http://time.tcx.org.uk/utc/2011-03-05/00:00
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1xNqIACgkQIXd7fCuc5vL22QCeM2IXEqx0x3yRrcbXIbjVbCsW
GR0An1hYPDoUq58rs1G9fSxgTrT4P1h1
=4BKf
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I'm currently copying user-store to a new disk array; this should be 
finished in a couple of days.  I hope the new array will be faster, as 
well as much larger.  There will be a separate maintenance notice for 
the actual switch later.

Once the move is done, in additional to having more storage now (about 
14TB), it will be much easier to add additional storage in the future.  
However, I don't think having a single 14+ TB filesystem is a good idea 
(the initial volume will be 5TB), so I'm considering new ways to split 
up the storage.

The most obvious is to have one volume for miscellaneous files, like the 
current user-store, and then one volume per large project: stats, dumps, 
etc.  People (or MMTs) who wanted additional storage for their project 
could request a new volume of a particular size.  As well as making it 
easier for us to account for space, this would mean you were guaranteed 
to get the amount of storage you asked for, unlike the current situation 
where the filesystem can (and does) fill up.

Does this seem like something that would be useful to users?  Otherwise, 
does anyone have another suggestions for how to allocate the space?

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1wAlwACgkQIXd7fCuc5vKrKACfXttPL1HS1EzLxykm0t5/dggn
BzkAnj3KiTWGs8QR55sb0/Jv5XQY0uie
=nN4u
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Maintenance notice: Tuesday, March 1

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> user-store will be unavailable until then.

user-store is now available again and the maintenance is complete.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1ubDIACgkQIXd7fCuc5vJK1wCfX8y/8+sXjfrbEjkCNqgfm07Z
imkAnRD01YyUYt8zUHw80MqAReffJpfp
=2SiZ
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] Maintenance notice: Tuesday, March 1

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

The database side of this is finished.  However, due to some problem 
with hemlock's disk array, the user-store filesystem is currently being 
fscked; this might take a few hours.  user-store will be unavailable 
until then.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1uVOMACgkQIXd7fCuc5vJJOwCfbGI9ipPKV8muhOjmeydSpOfa
ok4AoKx+q697F52E9gh2vbRCxhYniIJn
=foV6
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Maintenance notice: Tuesday, March 1

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> On the morning (UTC) of March 1 we will install some new hardware at the 
> Toolserver.  Services will be affected as follows:
> 
>  Service | Expected impact
>  +
>  Databases: s3, s4, s6, s7   | User databases unavailable for
>  |  < 10 minutes.
>  user-store filesystem   | Unavailable for < 10 minutes.

This maintenance has been postponed until tomorrow due to the 
unexpectedly long time it took to rack the new hardware.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1tMxEACgkQIXd7fCuc5vKXSgCeMehGP8eAxJQfS4y31co4ycBL
j+MAoIFyQkwjwUsWPaN9Og7RP96EMwmW
=Y8B1
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article ,
Magnus Manske   wrote:
> Thankfully, Marcin was actually helpful in answering this. Your
> attempt to distract from the actual issues at hand by "literal
> interpretation" has been duly noted.

I'm sorry you didn't find my post helpful, but I don't see any need for 
this sort of attitude.  I'm trying to help you.

> It's just that things like "SGE might interpret comments in your 
> scripts as commands and die" or "you can't run qsub from within qsub"  
> are types of behaviour that are unusual to say the least, and the 
> toolserver users are left to the mercy of these issues (I'd call them 
> bugs) without warning or documentation.

Recursive qsub not working was not a known issue until you posted it on 
the list just now, so it's unreasonable to complain about lack of 
documentation -- there's no way we can document an issue we don't even 
know about.

You are also the first person to have a problem with an unrelated #$ in 
a script.  Now that we can see that might be an issue, the documentation
can be updated.

The documentation is a wiki, so perhaps *you* could update it if you 
don't think it's very good.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1r0EwACgkQIXd7fCuc5vIXxwCgiDpQKCFf14bnWJrDyAujDIkr
P7gAn2TNI3NrIEOZUfm3/xvZ0pVMK48t
=1TEW
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[River Tarnell]

In article ,
Magnus Manske   wrote:
> Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo
> meaning from my scripts (which predate qsub availability) anyway?

Because it means you can include directives in your script indicating 
how it should run:

#! /bin/ksh
#$ -o $HOME/mytool.out
#$ -l h_rt=3:0:0
#$ -t 1-10
#$ -tc 3
# rest of script...

This is easier than having a separate script just to start that one.

> If that is desired for some cases, why is that activated by default?

With SGE, it's usually desirable for *all* cases (or at least the vast 
majority).  The cronsub script makes some of the most common uses 
unnecessary (e.g. options controlling output file), so it's used less 
there.  However, turning it off for cronsub while it was enabled for 
qsub seemed like it would be more confusing for users.

> Is there at least a qsub option to turn it off?

Yes.

> Will that option work with cronsub?

No.  But you can invoke qsub yourself if you prefer that.  (This is a 
known issue but it's somewhat difficult to fix because of how cronsub 
works.)
 
> In my "day job", I work with a large compute farm (lfs, ~10K cores),
> so I'm more than happy to work with, and migrate existing cronjobs to,
> this system, but it would help if it weren't a deliberate minefield...

I didn't write SGE.  I considered several existing batch scheduling 
systems and SGE seemed to be the best/most useful.  Of course that 
doesn't mean every user will like every feature it has.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[River Tarnell]

In article ,
Magnus Manske   wrote:
> > cronsub update_dupes /home/magnus/update_dupes.pl
> qsub: Unknown option
> 
> qsub: Unknown option
> 
> qsub: Unknown option

What is the output from this?

$ ksh -x /opt/local/bin/cronsub update_dupes /home/magnus/update_dupes.pl

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Maintenance notice: Tuesday, March 1

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

On the morning (UTC) of March 1 we will install some new hardware at the 
Toolserver.  Services will be affected as follows:

 Service | Expected impact
 +
 Databases: s3, s4, s6, s7   | User databases unavailable for
 |  < 10 minutes.
 user-store filesystem   | Unavailable for < 10 minutes.

Start time: Tuesday, 1st March, 1PM UTC (estimated)
http://time.tcx.org.uk/utc/2011/3/1/13/0

End time: Tuesday, 1st March, 4PM UTC (estimated)
http://time.tcx.org.uk/utc/2011/3/1/16/0

Details:

We will install new storage for databases and the user-store filesystem.  
This requires installing HBAs in three servers: hemlock, cassia, and 
hyacinth, which requires downtime.  We will try to stagger the 
installation of the two database servers so that those clusters 
remaining accessible during the maintenance, but user databases will be 
unavailable while hyacinth is down.

Before the maintenance on hemlock, we will forcibly unmount the 
user-store filesystem.  Any program accessing it at the time will 
receive an I/O error.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1ro3AACgkQIXd7fCuc5vLhsQCfbKKphnxKxXCM3VNCJC+YOsz+
E5sAn3EBKJjF1PeU4WZCT5W9KFNuqGqM
=TGse
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[River Tarnell]

In article ,
Magnus Manske   wrote:
> OK, so I gave up on starting my perl script using cronsub, which seems
> to work only if I give the perl script a parameter (WTF?), and instead
> altered it to be run manually, generating a shell script that could
> then start via cronsub.

This should not be the case.  Please provide the exact script and 
command line you used and the error message.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[River Tarnell]

In article ,
Marcin Cieslak   wrote:
> >> Magnus Manske  wrote:
> > I have a wrapper script that starts a handful of jobs on the
> > toolserver (via qsub). But, when I put that wrapper script as a
> > "normal" cronjob, I get some error messages about qsub not found.
> > Starting the wrapper via cronsub doesn't work either.

You should first run cronsub from the command-line with the same 
arguments.  If that still doesn't work, post the entire command line and 
any output from it.

Once you have it working from the command-line then it should be fine 
fromc ron.

> Actually cronsub does a litle bit more than sourcing
> /sge62/default/common/settings.sh file, so I don't know 
> why it's not working for you. Maybe you could share some output? 

cronsub makes it easier to submit longrun jobs, automatically sets the 
default output file, runs the script from $HOME and prevents duplicate 
jobs being submitted.  Otherwise it's identical to qsub.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to mail forwarding

[River Tarnell]

In article ,
Marcin Cieslak   wrote:
> I just tried emailing sa...@toolserver.org from both willow and nightshade,
> as well as externally. My external account gets the mail (all three emails)
> but none of the local accounts does (neither nightshade, willow, hawthorn,
> clematis, wolfsbane or ortelius). Is this because of my $HOME/.forward?

Under the current configuration, mail to u...@toolserver.org will never 
be sent to a local mailbox if you have an LDAP email address set.  This 
applies regardless of where the mail originates.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to mail forwarding

[River Tarnell]

In article ,
Marcin Cieslak   wrote:
> I am currently having "MYPRIMARYEMAILADDRESSOUTSIDE,\saper" in my 
> $HOME/.forward 
> which gives me a nice capability to use mail(1) or mailx(1) to quickly read
> whatever my jobs running on the localhost do. I do not mind 
> *not* having this replicated accross the cluster, so a process running
> on willow sends mail to willow etc. 
 
> I do not have any LDAP entry related to mail (and it's fine, too).
 
> Will local delivery on the Solaris boxes still possible?

Yes, this will still work.  However, since you have a .forward file, 
your LDAP email address will be entirely ignored.  This means that email 
to sa...@toolserver.org will be delivered to your local account on 
hawthorn or clematis as well as being forwarded to your forwarding 
address.  

If you have no email address set in LDAP, then nothing will change from 
how it works at the moment, except that incoming mail from outside the 
Toolserver will arrive on clematis/hawthorn instead of on nightshade.

> I understand that all hosts will start processing LDAP entries first,
> and forwarding the rest to the two hosts you named.

Not exactly. clematis and hawthorn will only handle incoming SMTP, i.e.  
mail to u...@toolserver.org... 

> From the point of redundancy, I'd rather have local mail delivered where
> it originated, since that host is most possibly up. I personally don't
> care much about mail originated externally (preferably I'd like
> to have one local mailbox to clean up).

... which mean that if you send mail from a Toolserver system, it will 
be delivered by that system, whether it's delivered locally or forwarded 
elsewhere.  This includes mail from SGE and from cron, and it applies 
even if you send mail to an @toolserver.org address.  hawthorn/clematis 
are only special because they will be listed as the mail exchangers for 
toolserver.org.

> Will local, per-host mail (from mail(1) or SGE) be preserved?

Yes, as above.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Changes to mail forwarding

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

During the next maintenance window (March 7th), we will make some 
changes to how mail forwarding works.  This will not affect most users, 
but you should read this mail anyway, especially if you make use of 
.forward files.  A summary of the changes you might need to make are at 
the end of this mail.

At present, all mail to u...@toolserver.org is sent to your LDAP alias; 
if you don't have an address set in LDAP, mail is sent to nightshade and 
processed locally (which might include using .forward files).  Locally 
generated mail, e.g. cron, is handled by .forward; users with email 
addresses set in LDAP have "usern...@toolserver.org" in $HOME/.forward 
so it's forwarded properly.

After this change, mail to u...@toolserver.org and mail to
u...@anymachine.toolserver.org will be handled as follows:

* If $HOME/.forward exists, mail will be delivered to the address in 
  that file;
* If not, mail will be delivered to the user's LDAP email address, if 
  any if set;
* If none is set, mail will be delivered locally on either clematis or 
  hawthorn (*not* nightshade).

In addition, mail to user+...@toolserver.org and 
user+...@anymachine.toolserver.org will be accepted, where "foo" can be 
any string.  This will be handled in the same way, except that 
$HOME/.forward+foo will be checked instead of $HOME/.forward.  This 
allows you to process mail differently depending on the address (see 
below for an example).

In summary, you should make these changes after the maintenance:

* If you do not do anything "special" with your mail, and you have an 
  LDAP mail address set, and your .forward file contains 
  "usern...@toolserver.org", then you do not need to do anything.  We 
  will delete your .forward file during the maintenance, since it's no 
  longer necessary.  This applies to the vast majority of users.

* If you have an LDAP email address set, and your .forward file contains 
  a different address, you should delete your .forward file after the 
  maintenance.  We will not do this for you.

* If you do not have an LDAP email address set, and you have a .forward 
  file containing an email address, you do not have to do anything.  
  However, you *should* run 'setmail' to set an LDAP email address, then
  delete your .forward file.

* If you do not have an LDAP email address set, and you do not have a 
  .forward file, then currently your mail will be delivered locally on 
  nightshade.  After the maintenance, your mail will be delivered 
  locally on either hawthorn or clematis.  We do not support this 
  configuration, and you should set an email address in LDAP so your mail 
  is forwarded.  However, if you insist on having mail delivered locally,
  you can continue to do so.

* If you currently rely on accepting mail to 
  user+...@machine.toolserver.org and use .forward+foo to funnel this 
  mail to a program, then you do not have to do anything.  However, you 
  *should* instead send mail to user+...@toolserver.org, which does not 
  rely on a particular machine being up.  Note that if you do this, the 
  incoming mail will be processed on either clematis or hawthorn.

The explanation of these changes is more complicated than the actual 
changes.  The result should be that mail processing is much simpler and 
more obvious to users.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1pIIQACgkQIXd7fCuc5vKOkACfYd3na6TlCLOpuOiXWDEVBB1y
sdkAnilzSrqxYCgL0WalfYlwSt68yf4U
=2El7
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Web server maintenance now

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> I'm about to do some maintenance on ortelius, one of the web servers.  
> This should not have any impact of users.

This is now complete.  I added more space for HTTP logs on ortelius, so 
the recent problem where the filesystem fills up and the web server 
breaks should be fixed.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1jzigACgkQIXd7fCuc5vJ3FgCdEzrWQ7NfVxi2WETwfCjRgDjA
BDcAn0kthP7nKD83tblq6tjfGqhcKdM7
=9ERo
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Web server maintenance now

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I'm about to do some maintenance on ortelius, one of the web servers.  
This should not have any impact of users.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1jx2kACgkQIXd7fCuc5vI7jQCgp23SaKkR/GLDRkt6U0xW1RqN
O3oAoI75koGwU1kVPcdkqe5SXmbjdEQV
=yQK1
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Python 3.2 installed

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Python 3.2 is now available on the login servers.  Anyone using Python 
3.1 should switch to 3.2, since 3.1 is likely to be removed at some 
point.  People using 2.7 (the default) are not affected.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1h9UMACgkQIXd7fCuc5vKhgQCgp2w9cADOZjxf+G0kvGbg3a3o
RN0AoKsiOHJRNe1BaGFd6zw3e47baUS7
=YhIE
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Unknown file

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article ,
Junaid P V   wrote:
> Today I can see an unknown file named 'core' with modification date
> 13/02/2011 at one of directory under public_html.
> It is around 233 MB in size. It has been more a week from my last login.

You can run 'pstack core' to see the program that generated the core 
file (probably by crashing).  If you can reproduce the issue that causes 
the crash, you might consider reporting it to the developers of the 
software.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1b5FEACgkQIXd7fCuc5vJE6QCeIBw6dlb5La1GPn78MQsYVwex
pi0An24p0zcpFQA2WtStiMDbNMDj8utb
=LNt/
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Job scheduling and mail receiving

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <4d57e888.7040...@surfeu.ch>,
Dr. Trigon  wrote:
> Makes sense and means I should redirect output from 'cronsub' to stdout
> (or maybe stderr) to have 'cronie' make mail this to me... ;)

That won't do anything, because cronsub doesn't produce any output 
(except in case of an error); it submits the job to SGE, then exits.  
The job output is handled by SGE, which writes it to the configured 
output file.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1X9fEACgkQIXd7fCuc5vJ3PwCfYVwdKM8saGU5pvZSaDs+99Hm
ba8AoLrBSPxpPyxomolsjYcO8KiJOdbN
=FCas
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Job scheduling and mail receiving

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <4d56a4cd.6020...@surfeu.ch>,
Dr. Trigon  wrote:
> But I was wondering how SGE can replace such a common tool like CRON 
> without havin same functionallity... :)

Well, SGE does not replace cron.  SGE is an additional tool which is 
available to users; cron is not going away (indeed, cron is required to 
use SGE, except for one-shot jobs).

We would like all users to use SGE for their tools, which requires that 
everything which cron does on its own should *also* be possible, and 
easy, with SGE; otherwise users will not bother converting.  However, 
this is an ongoing process and is obviously not complete.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1XJp8ACgkQIXd7fCuc5vIq8wCfV0T+TfolyRsG99FEvJrWXce8
mAcAoKHn2Ln22VUgMj79xIm+UZ5ANZ9J
=iPMK
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] ZWS configuration

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article , MZMcBride   
wrote:
> If you log in to a web server and look in /opt, that's where the
> configuration for zeus is stored, I believe.

The web server configuration is not accessible to users.

river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1UnW0ACgkQIXd7fCuc5vLO6QCfYI37w8TIS+VyuRwmQ3e142PY
1SIAn39iNakynZaq78RgBa2SsVkfDEGT
=ARVN
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] ZWS configuration

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <92048.20224.24006-8740-1034033847-1297358...@email.cz>,
Danny B.   wrote:
> is there ZWS configuration available to view somewhere?

No.
 
> I'm particularly looking for things like list and order of default 
> documents (directory indexes)

The current list of index files is:
index.html, index.htm, index.shtml, index.cgi, index.php, index.fcgi

You really shouldn't rely on that particular order, though.  There's no 
guarantee that the order won't change in future, and I can't imagine any 
situation where you need to have both "index.html" and "index.fcgi" (for 
example) in the same directory.

> and content types

A comprehensive list of type<>extension mappings is available at 
/etc/opt/ts/apache/2.2/mime.types.  If any of these are missing in ZWS, 
you can open a request in JIRA to have them added.

> but I think it would be useful to have such relevant configuration 
> settings which have direct influence on behavior written somewhere if 
> it's not yet

This usually just results in out-of-date or incorrect documentation on 
the wiki, rather than any useful information.  Major parts of the 
configuration (such as how PHP and FastCGI work) is already documented.

river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1UnMQACgkQIXd7fCuc5vJXTwCfZOLpABrt6rz5gbv6y/maQdeq
lNEAoJaPAaHcLwoFQgJiL0PpZSXdnz52
=FZzz
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Job scheduling and mail receiving

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <4d529b28.7030...@vyznev.net>,
Ilmari Karonen   wrote:
> On 02/09/2011 03:22 AM, River Tarnell wrote:
> > Or, if you only wanted to receive output on failure:
> >
> > #! /bin/ksh
> > f=$(mktemp)
> > trap "rm -f $f" 0
> > /path/to/mytool>$f 2>&1 ||<$f mailx -s "Tool output" $LOGNAME
 
> Just out of curiosity... is there a particular reason for the -f switch 
> to rm here, or are you just trying to be extra sure the tempfile gets 
> deleted?

Because:

* Without -f, rm behaves differently depending on whether the script is 
  run from an interactive terminal or not, so I always use -f to get 
  consistent behaviour; and

* It doesn't matter if the file was already deleted, so I don't want an 
  error message if rm fails.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1S++cACgkQIXd7fCuc5vLkOwCfUbDGfLxRJioXM2mVfX3dCqka
pXsAn29jjiirj4GmFAWQ5+PaIA92jrG5
=Ufaq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Job scheduling and mail receiving

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <4d516a08.5050...@surfeu.ch>,
Dr. Trigon  wrote:
> Just a short question in [1] is mentioned how to trigger a mail in case
> 
> * the job starts
> * the job finishes
> 
> what about
> 
> * the job print something to stdout like for CRON jobs (in the early
>   days... ;))

I don't think there's a built-in method of doing this.  You could use a 
wrapper script that did something like:

#! /bin/ksh
/path/to/mytool 2>&1 | mailx -s "Tool output" $LOGNAME

Or, if you only wanted to receive output on failure:

#! /bin/ksh
f=$(mktemp)
trap "rm -f $f" 0
/path/to/mytool >$f 2>&1 || <$f mailx -s "Tool output" $LOGNAME

Or only if the command generated output:

#! /bin/ksh
f=$(mktemp)
trap "rm -f $f" 0
/path/to/mytool >$f 2>&1
test -s $f && <$f mailx -s "Tool output" $LOGNAME

... etc.

I think you're the first person who's asked about this, but it seems 
like a useful feature, so I'll see if there's a way to make it easier.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk1R7FIACgkQIXd7fCuc5vKEPgCdEJj19ST9Kmz1jknYCtFSo09x
iH0AnRMSq1OPN4N+20/8MtsaxmjgO8Ac
=3lS2
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Maintenance for schema changes, now

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article ,
Tim Landscheidt   wrote:
> one obvious solution would be to treat the Toolserver database servers 
> as part of WMF's domain so they can use their SOPs (*1) to maintain 
> them instead of manually coordinating with the Toolserver admins (or 
> not).

I don't think that would be any easier.  Our database systems are 
completely different from theirs, so it would require all WMF admins to 
learn (and remember) a new procedure before they could change anything 
on their side.  Since they usually don't think to notify us of major 
changes in advance, I don't think they would be very happy with that.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk1RU9EACgkQIXd7fCuc5vIpCwCgsJoicthjjp4n3xNQxZ6UdKdk
BnAAn0EsQyZWlk11mIJxIZPFQAU/sH7u
=T5Rg
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Maintenance for schema changes, now

[River Tarnell]

In article ,
Tim Landscheidt   wrote:
> Given WMDE's recent fundraising changes which now allow sup-
> porting WMF directly, are there any plans to blur the clear
> distinction between WMF's farm and the Toolserver database
> servers with the accompanying headaches in the near future?

Not that I know of.  What sort of changes were you expecting?

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Maintenance for schema changes, now

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bryan Tong Minh:
> On Mon, Feb 7, 2011 at 7:43 AM, River Tarnell  wrote:
> > This was not announced in advance since we only learnt about the changes
> > about 10 minutes ago.  Sorry.
> Is this something we can do about in the future?

I've asked WMF (i.e., Tim) to let us know about schema changes in 
advance next time, so we can apply them with less service interruption.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEUEARECAAYFAk1QB8QACgkQIXd7fCuc5vKIoACgthUu5p73cQ4RSLoGzOTIEQlc
bO8AmJmM3RBX4bhyNeYBUVyiuG+Jwf0=
=RCsT
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Maintenance for schema changes, now

[River Tarnell]

Hi,

We're about to apply necessary schema changes for Wikimedia's MediaWiki
1.17 upgrade to the Toolserver databases.  We will apply the changes to
one server for each cluster at once; while the primary server is being
updated, user databases will be unavailable, but replication won't be
interrupted.

The exception is s2/s5, which currently only has one server.  Databases
on these clusters will not be replicated and might be partially
unavailable during the maintenance.

This was not announced in advance since we only learnt about the changes
about 10 minutes ago.  Sorry.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] A beginner's question

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Alex:
> Alex Brollo wrote:
>> I'd like to install into my toolserver account djvuLibre binaries.
> Or just file a JIRA ticket to have it installed for everyone.

It is already installed, and has been for some time:

willow% pkginfo|grep djvu
TS  TSdjvulibre DjVu libraries and utilities
willow% pkgchk -l TSdjvulibre | grep /bin
NOTE: Couldn't lock the package database.
Pathname: /opt/ts/bin
Pathname: /opt/ts/bin/any2djvu
Pathname: /opt/ts/bin/bzz
Pathname: /opt/ts/bin/c44
Pathname: /opt/ts/bin/cjb2
Pathname: /opt/ts/bin/cpaldjvu
...

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk1O7xwACgkQIXd7fCuc5vJHnACfTAqwCSNs763OeADAc9z2oYGV
5+IAnAo+NIjwzUrFu7yAoUahPsAdmSsj
=qUJ/
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] nagios test install

[River Tarnell]

In article ,
Mike Dupont   wrote:
>I would like to install nagious for testing, would that be ok? I am working
>on patches for nagios, and would like to monitor free software projects
>services for quality.

No, because:

* Nagios (the software) is a large web application and cannot be 
  installed according to the rules.

* Nagios (the project) is nothing to do with Wikimedia, and is not a 
  designated affiliate project of the Toolserver (again according to the 
  rules), so using the Toolserver for working on that project is not 
  allowed.

* "Free software projects", in general, are not designated affiliate 
  projects of the Toolserver either, so using the Toolserver for 
  anything related to those would be outside the rules as well.

More information: .

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Backing up a user database

[River Tarnell]

In article ,
Carl (CBM)  wrote:
>I realized I have to make some character set changes on one of my user
>databases, and I want to make a backup before I start. 

>I was planning to use mysqldump for this. Is there a better option
>that I'm overlooking?

Unlikely.

>Is there any reason that the output of 'mysqldump -hSERVER DATABASE'
>with no additional options would have problems restoring on the
>toolserver databases?

The only reason I've seen for this to fail is if you manually increase 
the limit on the size of MEMORY tables, then dump and reimport the 
tables; if the default limit is smaller, the import will fail.

I believe the maximum size of a MEMORY table is now the same as the 
default limit (to stop people using up massive amounts of memory), so 
this probably won't be an issue.

>Are there any subtle options that need to be set on mysqldump to make
>sure the output is usable in that environment?

In previous versions of MySQL it was necessary to use 'mysqldump 
--default-character-set=latin1' if you had tables marked as Latin-1 
which actually contained UTF-8 data.  Otherwise, the dump would be 
corrupted and unusable.

I think this has been fixed (since MySQL now has partial Unicode 
support), but it would probably not hurt to use it anyway if you have 
such tables.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] NNTP access to Toolserver lists

[River Tarnell]

Hi,

As I previously posted on wikitech-l, some Wikimedia lists are now
available via NNTP.  toolserver-l and toolserver-announce are included
in the initial selection of available lists; if you would like to read
them via NNTP (and, for toolserver-l, post), see
.

Compared to the GMane interface, this gateway:

* Does not rename lists (all lists are called wikimedia.),
* Does not rewrite email addresses in posts, thereby breaking PGP
  signatures.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] No maintenance Monday, 7th February

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

According to our maintenance schedule, February 7th would usually be a 
general maintenance.  However, due to the problems with s3/s7 we have 
not had enough time to plan it; there will therefore be no maintenance 
this time.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1Enq8ACgkQIXd7fCuc5vLrUQCfT7bJEpisyeVpZnNiEfhkM8nu
NcsAn0a7RiZd4ef+KgVem5WbbAiAlRtZ
=92od
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] phpMyAdmin

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

We have moved phpMyAdmin to a new host.  This should not affect users, 
but please open a request in JIRA if you find any odd problems with it.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0/GKEACgkQIXd7fCuc5vIgxACgmil7NmPpx2myeBdwQqWNwTOg
kjgAnjKzphlrKKVU8/8VLkfNPNrdMnbS
=RxHX
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] s3/s7 issues

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> * Several s3 databases (all of which start with 'a') are no longer 
>   available
> * s3 replication is halted due to the missing databases
> * s7 replication is halted to prevent further destruction of data.
 
> The only way to resolve this issue is to re-import the data from 
> Wikimedia's databases, which will take a few days at least.

This should be fixed now.  s3 is fine, while s7 is replicating but ~2.5 
days behind.  It should catch up within the next day or so.  Currently 
only hyacinth is serving s3, s4, s6 and s7; once s7 has caught up, we 
will copy its data to cassia and re-add the secondary server.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk06+w0ACgkQIXd7fCuc5vIpAACeISoKdFXcd69rQ8abstEgB67x
djUAoMIasSdu3LxQOuUa7zVOVdmIAtez
=+VX2
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] Out of schedule maintenance: s3/s4/s6/s7

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> Shortly we will move user databases on the s3/s4/s6/s7 server from the 
> current primary (cassia) to the secondary server (hyacinth).  hyacinth 
> will become the new primary server.  This will involve a couple of hours 
> read-only time for user databases during the move.

This has now been completed and user databases should be available 
again.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk03GtgACgkQIXd7fCuc5vKzUACgrJ8jMQWr7lq3ZFlHHhXUO7us
NrsAnjsVqouabnO8fWJL6eMUwcSa9+Qe
=0q4p
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Out of schedule maintenance: s3/s4/s6/s7

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Shortly we will move user databases on the s3/s4/s6/s7 server from the 
current primary (cassia) to the secondary server (hyacinth).  hyacinth 
will become the new primary server.  This will involve a couple of hours 
read-only time for user databases during the move.

Due to the current problems with s3/s7 this maintenance is being done 
outside the normal schedule.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk028soACgkQIXd7fCuc5vJH2gCggv1qjvygWzAM+sUVljUqtZ12
iG0An2fMIRacYeffR9o+hfJ13y8juocm
=vj5q
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] s3/s7 issues

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Bryan Tong Minh:
> Would it be possible to revoke the drop command from the replication
> user to prevent this from happening in the future?

Yes.  I also considered modifying trainwreck (our replication tool) to 
ignore DROP DATABASE commands; perhaps we could do both.

It would be easier if these commands wouldn't find their way into the 
binlog in the first place, but mistakes happen.  Unfortunately there's 
no way to prevent every possible command that might break our database.

In the future I hope to have one MySQL instance per cluster, which would 
more closely mirror Wikimedia's configuration and hopefully make errors 
like this less common.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk01/BgACgkQIXd7fCuc5vIkHQCfYdoPQPCoV52VEDPt290CepZf
gD0AoJ5vmv/wXth2Pldw1n3DvVbOV01j
=Nj8x
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] s3/s7 issues

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

After the s3/s7 split at Wikimedia, which moved several databases from 
s3 to the new s7 cluster, we retained both databases on the same (s3) 
server, which is our usual policy in such cases.  

Unfortunately, over the last couple of days Wikimedia executed several 
DROP DATABASE statements on their s7 server, for the old s3 databases.  
These statements were replicated to our s3/s7 server and dropped the 
live databases on our server.  As a result:

* Several s3 databases (all of which start with 'a') are no longer 
  available
* s3 replication is halted due to the missing databases
* s7 replication is halted to prevent further destruction of data.

The only way to resolve this issue is to re-import the data from 
Wikimedia's databases, which will take a few days at least.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk01800ACgkQIXd7fCuc5vKLMACeIN62eR9PXNDy475V9GLP3Edx
vggAmwUieFLO2hbF6S9NpJh/K3JsFx9G
=nI96
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] New job submission servers

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

We have set up a pair of redundant hosts to act as SGE job submission servers.  
These work by sharing each user's cronietab between both hosts, and executing 
jobs on whichever server is working.  This avoids that problem where jobs run 
from cronie on one login server (such as willow) will fail to run if that host 
is down, even when other login servers are available.

To use the new hosts, log into submit.toolserver.org and set up a cronietab 
(*not* a crontab) as normal.

Note that these hosts are *only* for submitting SGE jobs, not for running tools 
on.

For now this service could be considered experimental, but I don't foresee any 
problems with it.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0tewgACgkQIXd7fCuc5vJKpQCeOwDC6Y2/0rBkzGFj6A2WT1DS
aaQAn0yhdkmZikrbXPdPL26EsQ5A+r+1
=PDMJ
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] cronsub

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Dr. Trigon:
> 'qsub $HOME/pywikipedia/runbotrun.py -cron'
 
> which is the equivalent to:
 
> '0 2 * * * cronsub -s mainbot $HOME/pywikipedia/runbotrun.py -cron'
 
These are not equivalent.  cronsub does a fair amount of additional processing 
on top of qsub.

> in my cron(ie)tab file. But it seams that the change you mentioned 
> "Previously, SGE would copy the job script to a shared directory" does not 
> trigger for 'qsub'

That's correct.

> since I thought I can use 'qsub' to test my crontab entries... But may be I'm 
> wrong here.

You can run cronsub from the command-line for testing.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0o1Z8ACgkQIXd7fCuc5vIfYgCgm74Dowh/PT9e1Q9jvCFVXMl7
jRsAnR/kzcVHc5r4MQH1Tg7LUC5hIWW6
=ohhG
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] cronsub

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

seth:
> 0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/local/bin/cronsub -s
> seth-vdetector % /usr/bin/perl $HOME/bots/vdetector.pl >/dev/null
 
For a long-running job, you should use "-sl".  Otherwise your job will be 
killed after 6 hours.  (This is not documented very well on [[cron]] at the 
moment.)

> Base class package "Bot::BasicBot" is empty.

For software which was previously installed on nightshade but now isn't, you 
should file a request in the TS project in JIRA and include "[solaris]" in the 
subject.

> I asked about Basic::Bot more than 9 months ago, but did not get any
> answer. :-(
> https://wiki.toolserver.org/w/index.php?title=Conversion_of_nightshade_to_Solaris&action=historysubmit&diff=3685&oldid=3680
 
Sorry, but this page was never intended for reporting issues with the 
migration.  We asked users several times to test their tools and report issues 
in JIRA.

> /opt/local/bin/cronsub[38]: shift: bad number
 
> I guess, this has to do with river's change. But I don't know what to
> to, since $HOME/bots/vdetector.pl is already at -rwxr-xr-x.

Yes, I broke this earlier today.  It should be fixed now.  Sorry.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0oRZwACgkQIXd7fCuc5vKRYQCfaFUxvQCH6MKLBuaKfChN0nWT
jfkAn3feTEQDm1glPYRJryZizaEj/B9E
=cko2
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] cronsub

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I've made a couple of changes to cronsub in response to some issues reported by 
users.  Specifically:

* cronsub now requires that the script file be executable, and will raise an 
  error if it's not.  The previous behaviour was that non-executable scripts 
  would be executed by /bin/sh.  If this affects you, the fix is to make the 
  script executable (chmod +x).

* Previously, SGE would copy the job script to a shared directory and execute 
  it there.  For example, if you submitted $HOME/test.py, it would be copied 
  to a file such as /sge62/default/spool/wolfsbane/job_scripts/117333 before 
  starting.  This was unfortunate for Python users who depended on Python's 
  behaviour of treating the script's directory as part of the "import" search 
  path.

  This behaviour has been changed, so that the script file will be executed in 
  its original location.

I appreciate that the first item may be a breaking change for some users.  
However, on balance this seems like the lesser evil.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0ny8sACgkQIXd7fCuc5vK1UwCfaIt9gOSWym8sDOKR8bekuUaK
Qy4AoLLJIB1xaUjeKdpUHLZ6npg/0FiS
=VV97
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Job scheduling a python bot script

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Dr. Trigon:
> ... and this is the point were I am getting lost... this message
> looks VERY strange to me, since I'm not using an X server (explicitely)
> and I have no clue what the 'error/import.c' means there.

The problem is that your Python script is not executable, and is therefore run 
as a shell (/bin/sh) script.  "import" is a shell command from ImageMagick 
which takes a screenshot of an X display.  The fix is to make the script 
executable.

This behaviour is somewhat unfortunate and there's an open issue to resolve it, 
which will probably mean having cronsub raise an error if the script is not 
executable.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0nwtIACgkQIXd7fCuc5vLM6wCgwBN1yHu13cevCpInD11g0SP3
wvwAoJq3lQD8gTIfJhpzXCGpjqZX8Vqm
=Ckff
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] .environment file

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

It's now possible to set environment variables in your $HOME/.environment file, 
as a series of NAME=value pairs, for example:

PYTHONPATH=$HOME/pylib

This will affect interactive logins, cron jobs and SGE jobs, but currently not 
web server CGI scripts.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0mH1EACgkQIXd7fCuc5vJrWwCgt+nTHauHPBIJ2+jhw3J2tkPt
COcAnAurVJt1Wncm6WDM8eX/h0RkTz7+
=KJ0E
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Submitting Python jobs to SGE from cron

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Russell Blau:
> I was never able to get cronsub to accept arguments to the Python script; 

This seems to work fine for me:

willow% cat test.py
#!/usr/bin/python
#$ -j y
#$ -o $HOME/test.out
import sys
print sys.argv
willow% cronsub test test.py a b c
willow% tail ~/test.out
['/sge62/default/spool/wolfsbane/job_scripts/114704', 'a', 'b', 'c']

I think we fixed cronsub some time ago to allow arguments, both to qsub and the 
script itself.  (Did you ever report this problem to us?)

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0mEnkACgkQIXd7fCuc5vK6jACaAye7rfEapwJrB5rYLlf2lKZQ
p7gAnjFM/EudjCWo6GrMQKzibfzla2cR
=35y7
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Submitting Python jobs to SGE from cron

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Russell Blau:
> With these scripts in your home directly, you can then set up a cron job to 
> run any Python script using the following syntax in your crontab:
 
>  30  9   *   *   *$HOME/pysub jobname $HOME/path/script.py arg1 arg2

What feature(s) does this provide that cronsub doesn't?

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0mDFAACgkQIXd7fCuc5vJCaQCeO8xJQypYe6p4y08ijt7/2gl2
wngAn0CbmLG4nie6bNqPs7EDJYc8clZZ
=7wHw
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] cron

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Since the switch to Solaris a few people have complained about the Sun 'cron' 
not being very good.  As an alternative to cron, we have now installed 
'cronie', which is the RedHat Linux version of cron.  This is based on Vixie 
cron and offers an almost identical feature set to the old cron on 
nightshade.

To avoid a conflict with the system cron, the cronie crontab command is called 
'cronie'.  To convert your current crontab to a cronietab[0], you can use a 
command like this:

 % crontab -l >crontab  # list existing crontab to file
 % cronie crontab   # install same as a cronietab
 % crontab -r   # remove old crontab

You should probably use either crontab or cronie, but not both, since that 
could be confusing.

- river.

[0] Yes, "cronietab" is a fairly stupid name.  Sorry.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0lXhoACgkQIXd7fCuc5vKuEACdHkfvuZeTdT8MmMmNpgY1O/t6
7PwAn1tk+uJ9czNa+MBWQWP4vndzN+Et
=FIKF
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] jobserver

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Effectively immediately we are discontinuing support for the jobserver (that 
is, the "job" command) on willow.  The replacement for the jobserver is SGE:

  

If you current use the job server, please migrate your jobs to SGE and then 
delete them.  We will not shut down the job server until all users have 
migrated.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAk0lIyIACgkQIXd7fCuc5vLYxQCgkpBdrvdYiCBDe/4QR2Xx4tDU
K/AAn1LeTo77idvq4jYB72HbNeqrSwor
=je9C
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] Maintenance

[River Tarnell]

emijrp:
> 2011/1/3 River Tarnell 
> >  $ crontab $HOME/crontab.before_nightshade_reinstall
> I get errors http://toolserver.org/~emijrp/cronerrors.txt
 
> > You may want to review
> > <https://wiki.toolserver.org/view/Using_Solaris#cron> before installing
> > the new crontab.

Please read the page I linked to.

- river.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] Maintenance

[River Tarnell]

Note: crontabs from nightshade have been saved as 
$HOME/crontab.before_nightshade_reinstall.  You should review this for 
any necessary changes, then install it:

  $ crontab $HOME/crontab.before_nightshade_reinstall

Until you do this, your crontab will not run.

You may want to review 
 before installing 
the new crontab.

- river/

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette