Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s[3467]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Unfortunately this problem came back. I will set cassia to read-only, and move user databases to the secondary server (hyacinth) until the problem is fixed. This will cause an outage of user databases for a few hours. Since we're now using separate server instances for each cluster, databases will be imported as describe in [0]. - river. [0] http://lists.wikimedia.org/pipermail/toolserver-announce/2011-June/000440.html -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk44APQACgkQIXd7fCuc5vKRGgCeKWfizVDgfZoRjwPyKpfdTXLu ZSwAoJ41LbAoiBzNrDtp2+o1XAEXBqQa =SFMY -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s[3467]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, This seems to have been some kind of controller problem. After the power cycle, the server looks fine, and user databases should be available again. The server might be a bit slow since the RAID array is rebuilding. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk41fnYACgkQIXd7fCuc5vIFfQCgrViJyviWugNVn6PEKAANDZUc AukAn0/8kcxvm21txC8oSUnui9qCO466 =gpYs -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] SSH login issue
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jimmy Xu: It turns out that Toolserver doesn't support compression. I commented out Compression yes in my config and it started to work. This should be fixed now. The problem was that sshd does a chroot() in the privsep child, which handles communication with the client. However, Solaris ld.so does lazy-loading of shared libraries by default, and sshd doesn't require zlib until after the chroot. Since there is no zlib in the chroot directory, it failed to load: ld.so.1: sshd: fatal: libz.so.1: open failed: No such file or directory I have re-built sshd with lazy-loading disabled, and compression (ssh - -C) seems to be working now. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk4RuNIACgkQIXd7fCuc5vJ8NwCggwox2UqXzLqh+MaKvFnvgSTc P3MAoIiKGJ8Kb0nUqkTrkNv02MZx80lR =Vbz/ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] s3/s4/s6/s7 server split - action required for user databases
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, This mail affects you if you have user databases on s3, s4, s6 or s7. Other users can ignore it. Currently, these clusters are all on the same server. During the re-import, I will do an internal server split, i.e. each cluster will be on a different MySQL server afterwards. We will then move user databases to the new servers, and do the same thing on the other server. This presents a problem when restoring user databases, because it's not clear which cluster(s) the user intended each database to exist on. That problem will be resolved as follows: User databases which contain a wiki name or a cluster name in the database name will be restored to that cluster. All other databases will be restored to s3. If you don't agree with this algorithm, you should list your database name under the appropriate cluster at [0] and it will be restored to the server you request. You can list one database under multiple servers, if you want (but please consider disk space requirements before doing that). - river. [0] https://wiki.toolserver.org/view/User:River/hyacinth_db_split -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk4B3HEACgkQIXd7fCuc5vJrlQCeN3lbb/GRgzihawzBWobeIttn NAEAoLcyAN7eHNetAcpBjfH5gFEfi2BQ =B63B -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Re-importing of several database clusters
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm about to re-import several database clusters from WMF: s3, s4, s6 and s7. This will be done on the secondary server first, so users won't be affected, except that queries on these clusters might be a bit slower for a while. This will resolve the following TS issues: TS-948 creating of rue.wiki TS-926 Cassia and hyacinth are missing the ruewiki-database TS-1013 metawiki Database seems to be corrupt And might resolve the following issues, although they don't provide enough information to be sure of the cause: TS-1024 cswikisource_p needs reimport TS-1048 cswiktionary_p needs reimport This reimport might take longer than usual because I will be testing a new way of configuring database servers (which hopefully reduce database corruption in the future). Once this is done, I'll switch user databases to the secondary server (during which they will be offline for a couple of hours), and then reimport the other server. s1, s2 and s5 will be done later. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk4AxugACgkQIXd7fCuc5vIELACdF8k2zkgsakxSo6x+jyrySy2o KEwAnigGCyqS8td/t7gcQT97wjAwAOvV =citf -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Apache testing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I've set up an Apache instance on wolfsbane for testing. Please have a look at https://wiki.toolserver.org/view/Apache_testing and then test your tools (note the caveat about PHP, in particular). We're not in any rush to move off ZWS, so this is mostly just preliminary testing. Nonetheless, it would be useful to know about any problems now rather than later. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3/ahAACgkQIXd7fCuc5vI6+gCfe58t1WD4D1NYnQlACu5miPjV WUsAnR0jWJDB2bkXFQFl2qhnPUt3iDfX =EFB1 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Apache testing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mashiah Davidson: Much better than zws. No issues with database strings encoding, compare: http://wolfsbane.toolserver.org:81/~lvova/cgi-bin/go.sh?language=ruinterface=enlistby=zns http://toolserver.org/~lvova/cgi-bin/go.sh?language=ruinterface=enlistby=zns This is not an innate difference between the two, but some kind of configuration difference. I was looking at your encoding problem earlier; the most likely possibility seemed to be that after the MySQL 5.5 upgrade, the default client character set changed to UTF-8, which broke some tools. I've changed that back to latin1 to restore the old behaviour, but it didn't fix your problem. Looking at the difference in environment between the two: http://wolfsbane.toolserver.org:81/~river/env.cgi http://toolserver.org/~river/env.cgi ... the most obvious difference is that Apache doesn't set LANG=en_US.UTF-8. This doesn't seem to affect the MySQL client, though: river@wolfsbane:~$ LANG= sql enwiki_p ... Server characterset:latin1 Db characterset:latin1 Client characterset:latin1 Conn. characterset:latin1 river@wolfsbane:~$ LANG=en_US.UTF-8 sql enwiki_p ... Server characterset:latin1 Db characterset:latin1 Client characterset:latin1 Conn. characterset:latin1 If you could add the output of SHOW VARIABLES LIKE '%character_set%'; to the script, and see if there's a difference between them, it would help track down the problem. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3/sGoACgkQIXd7fCuc5vKi1gCgtXLmDP7Q6HSNv+q5tsH6fx/9 8zYAnjE1kHL4mctgrCTvYbfsPiccaI5B =1WLY -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Apache testing
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: I was looking at your encoding problem earlier; the most likely possibility seemed to be that after the MySQL 5.5 upgrade, the default client character set changed to UTF-8, which broke some tools. I've changed that back to latin1 to restore the old behaviour, but it didn't fix your problem. It looks like the problem is that you override --defaults-file in ~lvova/public_html/cgi-bin/ts. This prevents the client from reading /etc/my.cnf, so you don't get the correct default character set. If you do this instead: HOME=$(getent passwd $(id -u)|cut -f6 -d:) export HOME ... then you don't need to use --defaults-file and your script will probably work correct. (Example: http://toolserver.org/~river/sql.cgi.) - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3/s60ACgkQIXd7fCuc5vLYkwCglkXkbQjuKX8XeAuPaityD3kU X5MAn0dew3FwKcwDrTXRmdKCTMd+I1Oa =DlPv -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Maintenance on web servers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Either today or tomorrow I will be reinstalling the two web servers (wolfsbane and ortelius). This should not have any effect on users, and crontabs will be backed up and restored afterwards. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk37lngACgkQIXd7fCuc5vLK+ACgp1nGvciXLPNgbn5M3R9Zl4As b4gAn1tJaeFegx7JDO8MGFC1WuuwmYTL =Azzf -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] operating systems and web servers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Daniel Kinzler: why FreeBSD and not Linux? Why Linux and not FreeBSD, or Solaris, or OpenServer, or MP-RAS, or ...? I've asked this many times in the past (usually when someone says I want Linux) and never had a real answer that I can remember.[0] The way I see it, choice of operating system has very little impact on users because users don't, generally, interact with the operating system. There are a few specific cases which make a difference (I can think of cron, 'ps' and top/prstat), but apart from that, the software users actually use is the stuff in /opt/ts, which is independent of operating system.[1] For servers other than login servers (like databases) there are additional considerations, since those make a lot more use of OS features, like storage management. I've previously considered and rejected FreeBSD for use as a database server (in favour of staying with Solaris + VxVM). To me, FreeBSD seems to be an ideal candidate for a platform to layer /opt/ts on top of: it provides a base operating system which is reliable, has a good feature set in areas which only the OS can provide (e.g. DTrace, ZFS, auditing), and it's simple to install and maintain. - river. [0] I know some people are concerned about use of proprietary software on the Toolserver, but that's a separate issue, and in any case FreeBSD and Linux are equally open source. [1] This assumes that all the software we provide works on FreeBSD, of course. I haven't verified this, because we provide ~650 packages and it would take many hours, so there's no point doing it unless we actually decide to change OS. However, I don't foresee any problems. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3t5ikACgkQIXd7fCuc5vLokACfUbRk0fIc6yU6tdXaS+oTr26/ 0G8AoMGuJ149lizDh9KMO426taRJVDr8 =Hom2 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Change to toolserver.org HTTP
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I have switched toolserver.org HTTP to use a Squid reverse proxy for load-balancing instead of Solaris Cluster. In case of problems (like odd HTTP errors), please file an issue in JIRA. NB: This is unrelated to the previous IPv6 changes. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3kKkUACgkQIXd7fCuc5vK+JACglpYP1/yEV0k66fByrDT+Ztom x7QAmwXkPp9zXkhXRgIs6u4g5+ob4INh =vX+x -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] IPv6
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, During the next maintenance window I plan to start the (somewhat overdue) deployment of IPv6 at the Toolserver. In preparation for this, I will be adding IPv6 addresses to all hosts today. I don't expect this to cause any problems for users, except that the IP address for willow (which already has one) will change. I will leave the old IP in place until everything has migrated, but the new IP will be used for outgoing connections. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3h5w4ACgkQIXd7fCuc5vIc4QCggvuH0CpZ1scPjb82g8xjxcmX IkoAoK8xKuYW2WO8Zqv4T8qUbamZPAvQ =c2zi -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Changes to mail sending
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, At some point in the future we plan to make some changes to how mail sending on the Toolserver works. Unless you have tools which send mail (not including mail from cron or SGE), this change does not affect you. Specifically, we will no longer allow users to inject mail into the Toolserver mail system via SMTP, unless the mail is to a Toolserver address; in other words, internal hosts will be treated identically to any Internet host. This means you cannot send mail via SMTP to mail.toolserver.org or localhost. It will also be forbidden to send mail directly to Internet hosts, but this will not be enforced by technical means. If you send mail using /usr/lib/sendmail (or a comparable mechanism, such as /usr/bin/mail or Mutt), you do not need to do anything. (This includes PHP's mail() function.) If you send mail via SMTP, you should stop doing that, and instead use /usr/lib/sendmail. This generally means invoking /usr/lib/sendmail -oi -bm -- address and sending the mail body to it on stdin (including headers). Remember to escape any shell metacharacters in the address, if applicable. It should generally be trivial to convert anything that uses SMTP to use sendmail instead, and you should do this now rather than waiting. If you think this is not possible for some reason, you should let us know sooner rather than later. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3f5k8ACgkQIXd7fCuc5vLKwQCgnb388KrYECUXroxuogkART3p VMgAn1vZGwNyr7POKIKqP+DM3gxYpcN3 =7Dq0 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Change of SSL certificate for svn.toolserver.org
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, The SSL certificate for https://svn.toolserver.org has changed. The new fingerprint is available on https://fingerprints.toolserver.org/ and also below: SHA-1: E8:62:65:9A:89:CA:1C:0D:8B:97:80:93:F3:CA:04:F7:5F:B8:A8:D5 SHA-256: 4D:45:9B:60:E1:82:F6:57:4B:D0:EB:66:1C:22:25:21:95:24:2E:5A:7C:A6:C1:BC:B7:6D:FC:F4:6C:AF:84:E4 Your SVN client may require you to (re-)accept this key before you can access the repository. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3eYHMACgkQIXd7fCuc5vKW4ACdGqJNwbqxWfe8h828VNSkJjb/ pxAAoJ3hF/V8vC4G/wcsf/nBBVAVqOKd =ebwa -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Change to misc sites web server
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, The front-end web server on amaranth (which serves JIRA, MediaWiki, FishEye and WordPress) has been changed from Sun Java System Web Server to Apache. This should not cause any noticeable changes, but please report any problems to JIRA or ts-adm...@toolserver.org. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3T3OEACgkQIXd7fCuc5vJr8wCfYRN6DaltlbGWX+qfDv/R4oLe mSEAniY5Yfoj2VvWV1wsE+1Db8oWQb1/ =D6af -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Replag
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 John: Ive seen the primary sql servers for 1,2/5 lagged about 12 hours for the last day. One of the s1 servers (thyme) was lagging; probably because a user allocated a very large MEMORY table and caused the server to swap. I restarted and server (and fixed the limit on MEMORY table size), and it seems to be doing better now. I don't see any particularly serious lag for s2: http://munin.toolserver.org/Database/daphne/mysql_replication.html Unfortunately we don't have replication lag graphs for s5 at the moment. If there was lag there, it's probably because the s2/s5 server (daphne) is somewhat overloaded, since there's only one of them, while other clusters have two servers. While the fast servers are current. There are no fast servers anymore; sql-sX-rr and sql-sX-fast both point to both servers for each cluster (except s2/s5). - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3Q4egACgkQIXd7fCuc5vLq6ACfdx+WpdbVlVaE47qsFoV4grBV ShwAoKRMdQmVtOiD7EpVhs/YKHZc8et1 =8cOZ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Schema-only database backups
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, It is now possible to mark a user database to be backed up without its data, i.e. only the schema is dumped (mysqldump -d). To do this, include the string _transient in the database name: u_jsmith_transient u_jsmith_my_transient_data_p If you have databases whose data doesn't need to be backed up, it would be very helpful if you could rename them to include _transient in the name, to reduce the load / disk space requirements of the nightly backup job. Unfortunately MySQL doesn't provide a way to rename a database, but you can copy the database to a new name like this: $ mysqladmin create u_jsmith_transient $ mysqldump --opt u_jsmith | mysql u_jsmith_transient The old database should then be dropped. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3OGlQACgkQIXd7fCuc5vKdsQCaAwWY3LJ/H4+JcvB0x7VHTyzS VSMAnjVCQBvlfOVG/2DmLtUz0c9EUe5Y =sB8i -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] s1 servers changed
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: Due to high replication lag on thyme (s1) I've switched 'sql-s1' to rosemary. If you have scripts which access user databases on s1 and haven't been updated to use sql-s1-user instead, you will find your databases are missing. The fix is to connect to sql-s1-user instead of sql-s1. This has now been reverted. However, anyone who still connects to 'sql-s1' assuming it holds user databases should still change to using sql-s1-user. It's likely that we will eventually make this change permanent, to improve load distribution across the two servers (since many people still use sql-s1 instead of sql-s1-rr). - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3PTD8ACgkQIXd7fCuc5vJk5gCZAf1/R+xcDEzzYyykntIv3h1Z ElYAoJRbnx8dd6Se6WKS/1KzWda11cWh =lFTs -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] s1 servers changed
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Due to high replication lag on thyme (s1) I've switched 'sql-s1' to rosemary. If you have scripts which access user databases on s1 and haven't been updated to use sql-s1-user instead, you will find your databases are missing. The fix is to connect to sql-s1-user instead of sql-s1. Scripts which do not use user databases will not be affected and should not be changed. This change will be reverted once thyme has caught up. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk3OC2QACgkQIXd7fCuc5vIz7ACdENxlPUZHXengxHE/ZcYp4hkt g5oAoLTAXH4VJWJVZCla5sAhQ25sniRh =9Jc7 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Perl 5.14 now available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Perl 5.14.0-RC1 is now installed as /usr/bin/perl5.14. 5.14 will eventually become the default version of Perl; you may want to test your Perl programs with it before then. This is a pre-release version of Perl, so it may contain more bugs than usual. Please report any issues in JIRA. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk28ggkACgkQIXd7fCuc5vI9agCgws7XiUDPVlSsaOeDzae0jxcV eTIAnjChG53PvDAwBo93ux4FhS3fmHxU =3qSA -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Maintenance: JIRA, FishEye: Saturday 30th April
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: Tonight (UTC) we will upgrade JIRA and FishEye to the latest version. This will involve around 15 minutes downtime for each service. This is now finished. Downtime for JIRA was longer than expected (about 30 minutes) but within the maintenance window. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk27YJQACgkQIXd7fCuc5vIhPACeMA4ZquPHssQOdUc9vFUQFkDb DjwAn0TU8hU5cVGNM8s1SWJsHgj5VvNF =wC4e -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] PGP support for ts-admins address
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, The ts-admins contact address is now PGP-enabled. This means all mail coming this from address will be signed, and if you send encrypted email, you will receive encrypted email in reply. Notifications to admins are also encrypted, and the ticket system's web interface is accessed via SSL, so your mail should never be sent over the Internet in plaintext. However, mail is stored unencrypted in the database, so this is not secure if the ticket system itself is compromised. For this to work, your key needs to be available from the PGP keyserver network (the ticket system uses wwwkeys.uk.pgp.net). The key ID for ts-admins is: pub 4096R/E04DCDD7 2011-04-13 [expires: 2012-04-12] Key fingerprint = 5A5E 8634 DD8E DB05 F36E 3969 7035 1043 E04D CDD7 uid Wikimedia Toolserver ticket system accou...@toolserver.org uid Wikimedia Toolserver ticket system ts-adm...@toolserver.org ... which is available from wwwkeys, and is also listed at https://fingerprints.toolserver.org. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk2l9k8ACgkQIXd7fCuc5vIK3ACfbtDHqppbthS4PYQgMUeg/QSK GdoAoL2dhZOIgP6W1jWZ9JogIh/F/Fi1 =BIvL -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Alternative C function to vasprintf() on Toolserver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Andrew Dunbar:
Hmm currently I'm using _vscprintf / vsprintf on Windows/MSVC inside
#ifdef _WIN32
and vasprintf Ubuntu/gcc inside #else
But my C-fu is to puny to figure out what #ifdefs to use to detect
what combination of C compiler and libc I'm compiling under.
Apparently it's important with snprintf since some implementations
return an unspecified return value less than 1 with size=0.
That's true, but no current Unix implementation should do that, since
the return value of snprintf() is standardised in the current POSIX
standard (IEEE 1003.1-2004) and in the C99 standard (ISO/IEC 9899:1999).
The following code should handle Windows and both old and new Unix,
although it's completely untested and comes with no warranty:
/* Compile with -D_XOPEN_SOURCE=600 -std=c99 */
int
my_vasprintf(char **res, char const *fmt, va_list args)
{
int sz, r;
#ifdef _WIN32
sz = _vscprintf(fmt, args);
#else
sz = snprintf(NULL, 0, fmt, args);
#endif
#if defined(_WIN32) || (defined(__STDC__) __STDC__ = 199901L) \
|| (defined(_XOPEN_VERSION) (_XOPEN_VERSION = 600))
if (sz 0)
return sz;
if (sz = 0) {
#else
if (sz = 1) {
#endif
if ((*res = malloc(sz + 1)) == NULL)
return -1;
if ((sz = sprintf(*res, fmt, args)) 0) {
free(*res);
*res = NULL;
}
return sz;
}
#define MAXLN 65535
*res = NULL;
for (sz = 128; sz = MAXLN; sz *= 2) {
if ((*res = realloc(*res, sz)) == NULL)
return -1;
r = vsnprintf(*res, sz, fmt, args);
if (r 0 r sz)
return r;
}
errno = ENOMEM;
if (*res) {
free(*res);
*res = NULL;
}
return -1;
}
Obviously, this will be slower than the modern implementation, and you
may want to tune the initial buffer size for the expected string length.
- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (SunOS)
iEYEARECAAYFAk2i0QcACgkQIXd7fCuc5vI1ewCfTdlavg0rr56l7JzwAe5Oea9x
55sAnA4MIosJLfOnD/O3z/nkq0yqD5CE
=AGMe
-END PGP SIGNATURE-
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Alternative C function to vasprintf() on Toolserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew Dunbar: I've got a little program to index dump files that supports Windows and Linux but it doesn't compile on the Toolserver with either cc or gcc due to the lack of the function vasprintf(). It's a GNU extension so I'm surprised it didn't work even with gcc. Why doesn't the Toolserver gcc have it, and does anybody know of a workaround? *printf() is not part of the compiler, but the C library (libc); GNU glibc (used on Linux) provides it, while Solaris doesn't. There's been some discussion about it, but there are several incompatible asprintf() interfaces around and it's not clear which one should be implemented. Someone else already provided a version using snprintf, but you could also use g_vasprintf() from glib, or the version from gettext's libintl.h (which means you need to include libintl.h and link against - -lasprintf, both of which live in /opt/ts/gettext). I personally find the snprintf solution the best and this is what I use in my own code. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk2gaJ4ACgkQIXd7fCuc5vKziwCfTbDgK0nX32jK8iJcLWl278BK HNwAnRFW8/BhMUKS7zr6RRBcPlqCeWfj =31oq -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Changes to mail forwarding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 José Emilio Mori Recio: Regarding this [old] subject, and saying hello to everyone in my first mail here, it seems that $HOME/.forward (containing $h...@toolserver.org) should be no longer created for new accounts. After testing in my few-days-old account, external mail sent to j...@toolserver.org was going apparently to nowhere, until I deleted .forward and then began to arrive to my LDAP address, as expected... You're right, this file should not be created any more, and doing so will cause mail to be lost (actually, bounced, I hope). I've fixed mkuser to no longer create it and removed the bad file from recently created accounts. Thanks for reporting this -- no need to file an issue in JIRA. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (SunOS) iEYEARECAAYFAk2OMEAACgkQIXd7fCuc5vJn8wCfVNDG68inKHUTWePesX3tQ9KT OV4An0at+McdSuQGrmm6zsWWjNd5a09B =fBHO -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Schema changes, now
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, We're currently applying schema changes from WMF for a MediaWiki upgrade. To reduce replication lag, we will first apply them to an inactive server for each cluster, then switch users to that server and apply them on the other one. The first part has started now, and we will be switching in an hour or two. During the second part, user databases will be unavailable until we finish the changes and switch back. This may take several hours to finish. The exception is s2/s5, which only has one server; these clusters will be lagged during the change. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk11bNYACgkQIXd7fCuc5vIIowCgqKiJvPxtr+B19TRDrbJTy/70 fHQAn3CW69mS7tQPWrRxdMTKYnuFM5St =DpI/ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] New SGE execution host
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I've added yarrow as a login server and SGE execution host for all.q and longrun. All SGE jobs are now eligible to run there; no action is required from users. This server is for SGE only, tools must not be started there manually. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk11fv4ACgkQIXd7fCuc5vK24wCgjGrOQSwxLdndb3FeVbN/Dr4l FbQAn1GMC00MN8vDklXQqFBgDVv1mS7J =yQue -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] [SECURITY] SSH / Password login
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, During the maintenance on December 6th, 2010 I switched the Toolserver SSH server from Sun SSH to OpenSSH. A difference in how OpenSSH uses PAM to authenticate users meant that after the change, users were able to log via SSH using their LDAP password, without using an SSH key. This error has now been fixed. If you have no LDAP password set, or if you have a strong password[0], then this should not have affected you. However, if you had a weak or easily guessable password set, or if your LDAP password could have been compromised (e.g. if you wrote it down in plain text somewhere) then it's possible someone could have used it to gain access to your account. In that case, I suggest you immediately change your password (via 'passwd'), then review your home directory to ensure no unauthorised changes have been made (e.g. new SSH keys added, or shell rc files changed). If you have sensitive data such as SSH or PGP keys on the Toolserver, you may wish to revoke them and issue new ones. (However, storing that kind of data on the Toolserver is probably a bad idea in any case.) I'm very sorry for the inconvenience this issue might cause to users, and I will be reviewing our authentication configuration to reduce the chance of something like this happening in the future. - river. [0] Which is somewhat enforced by the LDAP password policy, but it's still possible to set a weak password if you try hard enough. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1zvOgACgkQIXd7fCuc5vJpowCeMoLig31BAHnStWakKgeU/ZOr pCYAoKMEF/6+yzzKGQNVYxXqJuhM2f63 =ykB1 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [SECURITY] SSH / Password login
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article slrnin7g7n.15f3.sa...@saper.info, Marcin Cieslak sa...@saper.info wrote: If you have no LDAP password set, (...) How do I find out if I have one? I don't remember I ever set this, or may be it was long ago. Run setpass: willow% setpass setpass: password already set setpass: use passwd(1) to change your password - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1zwhoACgkQIXd7fCuc5vImCQCguZ5c4S6yuVxuzSnK04gl64/G Yn0An389qvaqjjwKd+2vpOw2a3o5cgJ4 =GoHp -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [SECURITY] SSH / Password login
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article 4d73dec0.9030...@gmail.com, Platonides platoni...@gmail.com wrote: Wouldn't such login have been logged? Seems easy to find out if any account was accessed this way. No, because the problem was introduced in December and we don't keep 3 months' worth of old logs around. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1z3lIACgkQIXd7fCuc5vIokQCeMOScxP+wvLasspBfas+HQ5yA 09UAn1eyDgwbhy03C6jM3w5/TW/MRmPF =wU1u -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance tonight: mail service
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: Tonight I will implement the mail forwarding changes described here: http://lists.wikimedia.org/pipermail/toolserver-announce/2011-February/000403.html. This is now done. Please check everything works as you expect, and file any problems in JIRA. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk10MbEACgkQIXd7fCuc5vIS7QCfVmKcARn/NX1yeePVaHWqXtWz sMMAnjQagfEffogHL0KlK1ui3UtspCCD =BjHE -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] user-store maintenance, tonight
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Later tonight (around 12AM UTC[0]) I will restart the NFS server on hemlock, which serves user-store. This will cause an interruption to service that should last for under 1 minute. - river. [0] http://time.tcx.org.uk/utc/2011-03-05/00:00 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1xNqIACgkQIXd7fCuc5vL22QCeM2IXEqx0x3yRrcbXIbjVbCsW GR0An1hYPDoUq58rs1G9fSxgTrT4P1h1 =4BKf -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] user-store
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm currently copying user-store to a new disk array; this should be finished in a couple of days. I hope the new array will be faster, as well as much larger. There will be a separate maintenance notice for the actual switch later. Once the move is done, in additional to having more storage now (about 14TB), it will be much easier to add additional storage in the future. However, I don't think having a single 14+ TB filesystem is a good idea (the initial volume will be 5TB), so I'm considering new ways to split up the storage. The most obvious is to have one volume for miscellaneous files, like the current user-store, and then one volume per large project: stats, dumps, etc. People (or MMTs) who wanted additional storage for their project could request a new volume of a particular size. As well as making it easier for us to account for space, this would mean you were guaranteed to get the amount of storage you asked for, unlike the current situation where the filesystem can (and does) fill up. Does this seem like something that would be useful to users? Otherwise, does anyone have another suggestions for how to allocate the space? - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1wAlwACgkQIXd7fCuc5vKrKACfXttPL1HS1EzLxykm0t5/dggn BzkAnj3KiTWGs8QR55sb0/Jv5XQY0uie =nN4u -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance notice: Tuesday, March 1
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: user-store will be unavailable until then. user-store is now available again and the maintenance is complete. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1ubDIACgkQIXd7fCuc5vJK1wCfX8y/8+sXjfrbEjkCNqgfm07Z imkAnRD01YyUYt8zUHw80MqAReffJpfp =2SiZ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance notice: Tuesday, March 1
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: On the morning (UTC) of March 1 we will install some new hardware at the Toolserver. Services will be affected as follows: Service | Expected impact + Databases: s3, s4, s6, s7 | User databases unavailable for | 10 minutes. user-store filesystem | Unavailable for 10 minutes. This maintenance has been postponed until tomorrow due to the unexpectedly long time it took to rack the new hardware. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1tMxEACgkQIXd7fCuc5vKXSgCeMehGP8eAxJQfS4y31co4ycBL j+MAoIFyQkwjwUsWPaN9Og7RP96EMwmW =Y8B1 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
In article aanlktiknfjfxqdvgzcj63hz90v_xpmkqvu145yj+p...@mail.gmail.com, Magnus Manske magnusman...@googlemail.com wrote: OK, so I gave up on starting my perl script using cronsub, which seems to work only if I give the perl script a parameter (WTF?), and instead altered it to be run manually, generating a shell script that could then start via cronsub. This should not be the case. Please provide the exact script and command line you used and the error message. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Maintenance notice: Tuesday, March 1
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On the morning (UTC) of March 1 we will install some new hardware at the Toolserver. Services will be affected as follows: Service | Expected impact + Databases: s3, s4, s6, s7 | User databases unavailable for | 10 minutes. user-store filesystem | Unavailable for 10 minutes. Start time: Tuesday, 1st March, 1PM UTC (estimated) http://time.tcx.org.uk/utc/2011/3/1/13/0 End time: Tuesday, 1st March, 4PM UTC (estimated) http://time.tcx.org.uk/utc/2011/3/1/16/0 Details: We will install new storage for databases and the user-store filesystem. This requires installing HBAs in three servers: hemlock, cassia, and hyacinth, which requires downtime. We will try to stagger the installation of the two database servers so that those clusters remaining accessible during the maintenance, but user databases will be unavailable while hyacinth is down. Before the maintenance on hemlock, we will forcibly unmount the user-store filesystem. Any program accessing it at the time will receive an I/O error. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1ro3AACgkQIXd7fCuc5vLhsQCfbKKphnxKxXCM3VNCJC+YOsz+ E5sAn3EBKJjF1PeU4WZCT5W9KFNuqGqM =TGse -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Cronjob starting cronsub
In article slrnimkfjp.fte.sa...@saper.info, Marcin Cieslak sa...@saper.info wrote: Magnus Manske magnusman...@googlemail.com wrote: I have a wrapper script that starts a handful of jobs on the toolserver (via qsub). But, when I put that wrapper script as a normal cronjob, I get some error messages about qsub not found. Starting the wrapper via cronsub doesn't work either. You should first run cronsub from the command-line with the same arguments. If that still doesn't work, post the entire command line and any output from it. Once you have it working from the command-line then it should be fine fromc ron. Actually cronsub does a litle bit more than sourcing /sge62/default/common/settings.sh file, so I don't know why it's not working for you. Maybe you could share some output? cronsub makes it easier to submit longrun jobs, automatically sets the default output file, runs the script from $HOME and prevents duplicate jobs being submitted. Otherwise it's identical to qsub. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Changes to mail forwarding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, During the next maintenance window (March 7th), we will make some changes to how mail forwarding works. This will not affect most users, but you should read this mail anyway, especially if you make use of .forward files. A summary of the changes you might need to make are at the end of this mail. At present, all mail to u...@toolserver.org is sent to your LDAP alias; if you don't have an address set in LDAP, mail is sent to nightshade and processed locally (which might include using .forward files). Locally generated mail, e.g. cron, is handled by .forward; users with email addresses set in LDAP have usern...@toolserver.org in $HOME/.forward so it's forwarded properly. After this change, mail to u...@toolserver.org and mail to u...@anymachine.toolserver.org will be handled as follows: * If $HOME/.forward exists, mail will be delivered to the address in that file; * If not, mail will be delivered to the user's LDAP email address, if any if set; * If none is set, mail will be delivered locally on either clematis or hawthorn (*not* nightshade). In addition, mail to user+...@toolserver.org and user+...@anymachine.toolserver.org will be accepted, where foo can be any string. This will be handled in the same way, except that $HOME/.forward+foo will be checked instead of $HOME/.forward. This allows you to process mail differently depending on the address (see below for an example). In summary, you should make these changes after the maintenance: * If you do not do anything special with your mail, and you have an LDAP mail address set, and your .forward file contains usern...@toolserver.org, then you do not need to do anything. We will delete your .forward file during the maintenance, since it's no longer necessary. This applies to the vast majority of users. * If you have an LDAP email address set, and your .forward file contains a different address, you should delete your .forward file after the maintenance. We will not do this for you. * If you do not have an LDAP email address set, and you have a .forward file containing an email address, you do not have to do anything. However, you *should* run 'setmail' to set an LDAP email address, then delete your .forward file. * If you do not have an LDAP email address set, and you do not have a .forward file, then currently your mail will be delivered locally on nightshade. After the maintenance, your mail will be delivered locally on either hawthorn or clematis. We do not support this configuration, and you should set an email address in LDAP so your mail is forwarded. However, if you insist on having mail delivered locally, you can continue to do so. * If you currently rely on accepting mail to user+...@machine.toolserver.org and use .forward+foo to funnel this mail to a program, then you do not have to do anything. However, you *should* instead send mail to user+...@toolserver.org, which does not rely on a particular machine being up. Note that if you do this, the incoming mail will be processed on either clematis or hawthorn. The explanation of these changes is more complicated than the actual changes. The result should be that mail processing is much simpler and more obvious to users. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1pIIQACgkQIXd7fCuc5vKOkACfYd3na6TlCLOpuOiXWDEVBB1y sdkAnilzSrqxYCgL0WalfYlwSt68yf4U =2El7 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Changes to mail forwarding
In article slrnimil5m.fte.sa...@saper.info, Marcin Cieslak sa...@saper.info wrote: I just tried emailing sa...@toolserver.org from both willow and nightshade, as well as externally. My external account gets the mail (all three emails) but none of the local accounts does (neither nightshade, willow, hawthorn, clematis, wolfsbane or ortelius). Is this because of my $HOME/.forward? Under the current configuration, mail to u...@toolserver.org will never be sent to a local mailbox if you have an LDAP email address set. This applies regardless of where the mail originates. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Web server maintenance now
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm about to do some maintenance on ortelius, one of the web servers. This should not have any impact of users. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1jx2kACgkQIXd7fCuc5vI7jQCgp23SaKkR/GLDRkt6U0xW1RqN O3oAoI75koGwU1kVPcdkqe5SXmbjdEQV =yQK1 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Unknown file
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article AANLkTi=jgaur3piqo9eotrjg_emik_zhe+q34n1sf...@mail.gmail.com, Junaid P V junu...@gmail.com wrote: Today I can see an unknown file named 'core' with modification date 13/02/2011 at one of directory under public_html. It is around 233 MB in size. It has been more a week from my last login. You can run 'pstack core' to see the program that generated the core file (probably by crashing). If you can reproduce the issue that causes the crash, you might consider reporting it to the developers of the software. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1b5FEACgkQIXd7fCuc5vJE6QCeIBw6dlb5La1GPn78MQsYVwex pi0An24p0zcpFQA2WtStiMDbNMDj8utb =LNt/ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Job scheduling and mail receiving
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article 4d56a4cd.6020...@surfeu.ch, Dr. Trigon dr.tri...@surfeu.ch wrote: But I was wondering how SGE can replace such a common tool like CRON without havin same functionallity... :) Well, SGE does not replace cron. SGE is an additional tool which is available to users; cron is not going away (indeed, cron is required to use SGE, except for one-shot jobs). We would like all users to use SGE for their tools, which requires that everything which cron does on its own should *also* be possible, and easy, with SGE; otherwise users will not bother converting. However, this is an ongoing process and is obviously not complete. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1XJp8ACgkQIXd7fCuc5vIq8wCfV0T+TfolyRsG99FEvJrWXce8 mAcAoKHn2Ln22VUgMj79xIm+UZ5ANZ9J =iPMK -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] ZWS configuration
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article c97a0619.f26...@mzmcbride.com, MZMcBride z...@mzmcbride.com wrote: If you log in to a web server and look in /opt, that's where the configuration for zeus is stored, I believe. The web server configuration is not accessible to users. river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk1UnW0ACgkQIXd7fCuc5vLO6QCfYI37w8TIS+VyuRwmQ3e142PY 1SIAn39iNakynZaq78RgBa2SsVkfDEGT =ARVN -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance for schema changes, now
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article m3hbcesjx1@passepartout.tim-landscheidt.de, Tim Landscheidt t...@tim-landscheidt.de wrote: one obvious solution would be to treat the Toolserver database servers as part of WMF's domain so they can use their SOPs (*1) to maintain them instead of manually coordinating with the Toolserver admins (or not). I don't think that would be any easier. Our database systems are completely different from theirs, so it would require all WMF admins to learn (and remember) a new procedure before they could change anything on their side. Since they usually don't think to notify us of major changes in advance, I don't think they would be very happy with that. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1RU9EACgkQIXd7fCuc5vIpCwCgsJoicthjjp4n3xNQxZ6UdKdk BnAAn0EsQyZWlk11mIJxIZPFQAU/sH7u =T5Rg -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Job scheduling and mail receiving
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article 4d516a08.5050...@surfeu.ch, Dr. Trigon dr.tri...@surfeu.ch wrote: Just a short question in [1] is mentioned how to trigger a mail in case * the job starts * the job finishes what about * the job print something to stdout like for CRON jobs (in the early days... ;)) I don't think there's a built-in method of doing this. You could use a wrapper script that did something like: #! /bin/ksh /path/to/mytool 21 | mailx -s Tool output $LOGNAME Or, if you only wanted to receive output on failure: #! /bin/ksh f=$(mktemp) trap rm -f $f 0 /path/to/mytool $f 21 || $f mailx -s Tool output $LOGNAME Or only if the command generated output: #! /bin/ksh f=$(mktemp) trap rm -f $f 0 /path/to/mytool $f 21 test -s $f $f mailx -s Tool output $LOGNAME ... etc. I think you're the first person who's asked about this, but it seems like a useful feature, so I'll see if there's a way to make it easier. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1R7FIACgkQIXd7fCuc5vKEPgCdEJj19ST9Kmz1jknYCtFSo09x iH0AnRMSq1OPN4N+20/8MtsaxmjgO8Ac =3lS2 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance for schema changes, now
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bryan Tong Minh: On Mon, Feb 7, 2011 at 7:43 AM, River Tarnell ri...@yvaine.tcx.org.uk wrote: This was not announced in advance since we only learnt about the changes about 10 minutes ago. Sorry. Is this something we can do about in the future? I've asked WMF (i.e., Tim) to let us know about schema changes in advance next time, so we can apply them with less service interruption. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEUEARECAAYFAk1QB8QACgkQIXd7fCuc5vKIoACgthUu5p73cQ4RSLoGzOTIEQlc bO8AmJmM3RBX4bhyNeYBUVyiuG+Jwf0= =RCsT -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Maintenance for schema changes, now
In article m339nzkg9a@passepartout.tim-landscheidt.de, Tim Landscheidt t...@tim-landscheidt.de wrote: Given WMDE's recent fundraising changes which now allow sup- porting WMF directly, are there any plans to blur the clear distinction between WMF's farm and the Toolserver database servers with the accompanying headaches in the near future? Not that I know of. What sort of changes were you expecting? - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] A beginner's question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex: Alex Brollo wrote: I'd like to install into my toolserver account djvuLibre binaries. Or just file a JIRA ticket to have it installed for everyone. It is already installed, and has been for some time: willow% pkginfo|grep djvu TS TSdjvulibre DjVu libraries and utilities willow% pkgchk -l TSdjvulibre | grep /bin NOTE: Couldn't lock the package database. Pathname: /opt/ts/bin Pathname: /opt/ts/bin/any2djvu Pathname: /opt/ts/bin/bzz Pathname: /opt/ts/bin/c44 Pathname: /opt/ts/bin/cjb2 Pathname: /opt/ts/bin/cpaldjvu ... - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk1O7xwACgkQIXd7fCuc5vJHnACfTAqwCSNs763OeADAc9z2oYGV 5+IAnAo+NIjwzUrFu7yAoUahPsAdmSsj =qUJ/ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Maintenance for schema changes, now
Hi, We're about to apply necessary schema changes for Wikimedia's MediaWiki 1.17 upgrade to the Toolserver databases. We will apply the changes to one server for each cluster at once; while the primary server is being updated, user databases will be unavailable, but replication won't be interrupted. The exception is s2/s5, which currently only has one server. Databases on these clusters will not be replicated and might be partially unavailable during the maintenance. This was not announced in advance since we only learnt about the changes about 10 minutes ago. Sorry. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] NNTP access to Toolserver lists
Hi, As I previously posted on wikitech-l, some Wikimedia lists are now available via NNTP. toolserver-l and toolserver-announce are included in the initial selection of available lists; if you would like to read them via NNTP (and, for toolserver-l, post), see http://news.tcx.org.uk/wikimedia.html. Compared to the GMane interface, this gateway: * Does not rename lists (all lists are called wikimedia.list name), * Does not rewrite email addresses in posts, thereby breaking PGP signatures. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] phpMyAdmin
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, We have moved phpMyAdmin to a new host. This should not affect users, but please open a request in JIRA if you find any odd problems with it. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0/GKEACgkQIXd7fCuc5vIgxACgmil7NmPpx2myeBdwQqWNwTOg kjgAnjKzphlrKKVU8/8VLkfNPNrdMnbS =RxHX -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Out of schedule maintenance: s3/s4/s6/s7
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: Shortly we will move user databases on the s3/s4/s6/s7 server from the current primary (cassia) to the secondary server (hyacinth). hyacinth will become the new primary server. This will involve a couple of hours read-only time for user databases during the move. This has now been completed and user databases should be available again. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk03GtgACgkQIXd7fCuc5vKzUACgrJ8jMQWr7lq3ZFlHHhXUO7us NrsAnjsVqouabnO8fWJL6eMUwcSa9+Qe =0q4p -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] s3/s7 issues
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bryan Tong Minh: Would it be possible to revoke the drop command from the replication user to prevent this from happening in the future? Yes. I also considered modifying trainwreck (our replication tool) to ignore DROP DATABASE commands; perhaps we could do both. It would be easier if these commands wouldn't find their way into the binlog in the first place, but mistakes happen. Unfortunately there's no way to prevent every possible command that might break our database. In the future I hope to have one MySQL instance per cluster, which would more closely mirror Wikimedia's configuration and hopefully make errors like this less common. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk01/BgACgkQIXd7fCuc5vIkHQCfYdoPQPCoV52VEDPt290CepZf gD0AoJ5vmv/wXth2Pldw1n3DvVbOV01j =Nj8x -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] cronsub
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 seth: 0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/local/bin/cronsub -s seth-vdetector % /usr/bin/perl $HOME/bots/vdetector.pl /dev/null For a long-running job, you should use -sl. Otherwise your job will be killed after 6 hours. (This is not documented very well on [[cron]] at the moment.) Base class package Bot::BasicBot is empty. For software which was previously installed on nightshade but now isn't, you should file a request in the TS project in JIRA and include [solaris] in the subject. I asked about Basic::Bot more than 9 months ago, but did not get any answer. :-( https://wiki.toolserver.org/w/index.php?title=Conversion_of_nightshade_to_Solarisaction=historysubmitdiff=3685oldid=3680 Sorry, but this page was never intended for reporting issues with the migration. We asked users several times to test their tools and report issues in JIRA. /opt/local/bin/cronsub[38]: shift: bad number I guess, this has to do with river's change. But I don't know what to to, since $HOME/bots/vdetector.pl is already at -rwxr-xr-x. Yes, I broke this earlier today. It should be fixed now. Sorry. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0oRZwACgkQIXd7fCuc5vKRYQCfaFUxvQCH6MKLBuaKfChN0nWT jfkAn3feTEQDm1glPYRJryZizaEj/B9E =cko2 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] cronsub
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dr. Trigon: 'qsub $HOME/pywikipedia/runbotrun.py -cron' which is the equivalent to: '0 2 * * * cronsub -s mainbot $HOME/pywikipedia/runbotrun.py -cron' These are not equivalent. cronsub does a fair amount of additional processing on top of qsub. in my cron(ie)tab file. But it seams that the change you mentioned Previously, SGE would copy the job script to a shared directory does not trigger for 'qsub' That's correct. since I thought I can use 'qsub' to test my crontab entries... But may be I'm wrong here. You can run cronsub from the command-line for testing. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0o1Z8ACgkQIXd7fCuc5vIfYgCgm74Dowh/PT9e1Q9jvCFVXMl7 jRsAnR/kzcVHc5r4MQH1Tg7LUC5hIWW6 =ohhG -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Job scheduling a python bot script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dr. Trigon: ... and this is the point were I am getting lost... this message looks VERY strange to me, since I'm not using an X server (explicitely) and I have no clue what the 'error/import.c' means there. The problem is that your Python script is not executable, and is therefore run as a shell (/bin/sh) script. import is a shell command from ImageMagick which takes a screenshot of an X display. The fix is to make the script executable. This behaviour is somewhat unfortunate and there's an open issue to resolve it, which will probably mean having cronsub raise an error if the script is not executable. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0nwtIACgkQIXd7fCuc5vLM6wCgwBN1yHu13cevCpInD11g0SP3 wvwAoJq3lQD8gTIfJhpzXCGpjqZX8Vqm =Ckff -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] cronsub
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I've made a couple of changes to cronsub in response to some issues reported by users. Specifically: * cronsub now requires that the script file be executable, and will raise an error if it's not. The previous behaviour was that non-executable scripts would be executed by /bin/sh. If this affects you, the fix is to make the script executable (chmod +x). * Previously, SGE would copy the job script to a shared directory and execute it there. For example, if you submitted $HOME/test.py, it would be copied to a file such as /sge62/default/spool/wolfsbane/job_scripts/117333 before starting. This was unfortunate for Python users who depended on Python's behaviour of treating the script's directory as part of the import search path. This behaviour has been changed, so that the script file will be executed in its original location. I appreciate that the first item may be a breaking change for some users. However, on balance this seems like the lesser evil. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0ny8sACgkQIXd7fCuc5vK1UwCfaIt9gOSWym8sDOKR8bekuUaK Qy4AoLLJIB1xaUjeKdpUHLZ6npg/0FiS =VV97 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Submitting Python jobs to SGE from cron
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Russell Blau: With these scripts in your home directly, you can then set up a cron job to run any Python script using the following syntax in your crontab: 30 9 * * *$HOME/pysub jobname $HOME/path/script.py arg1 arg2 What feature(s) does this provide that cronsub doesn't? - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0mDFAACgkQIXd7fCuc5vJCaQCeO8xJQypYe6p4y08ijt7/2gl2 wngAn0CbmLG4nie6bNqPs7EDJYc8clZZ =7wHw -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Submitting Python jobs to SGE from cron
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Russell Blau: I was never able to get cronsub to accept arguments to the Python script; This seems to work fine for me: willow% cat test.py #!/usr/bin/python #$ -j y #$ -o $HOME/test.out import sys print sys.argv willow% cronsub test test.py a b c willow% tail ~/test.out ['/sge62/default/spool/wolfsbane/job_scripts/114704', 'a', 'b', 'c'] I think we fixed cronsub some time ago to allow arguments, both to qsub and the script itself. (Did you ever report this problem to us?) - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0mEnkACgkQIXd7fCuc5vK6jACaAye7rfEapwJrB5rYLlf2lKZQ p7gAnjFM/EudjCWo6GrMQKzibfzla2cR =35y7 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] .environment file
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, It's now possible to set environment variables in your $HOME/.environment file, as a series of NAME=value pairs, for example: PYTHONPATH=$HOME/pylib This will affect interactive logins, cron jobs and SGE jobs, but currently not web server CGI scripts. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0mH1EACgkQIXd7fCuc5vJrWwCgt+nTHauHPBIJ2+jhw3J2tkPt COcAnAurVJt1Wncm6WDM8eX/h0RkTz7+ =KJ0E -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] jobserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Effectively immediately we are discontinuing support for the jobserver (that is, the job command) on willow. The replacement for the jobserver is SGE: https://wiki.toolserver.org/view/Job_scheduling If you current use the job server, please migrate your jobs to SGE and then delete them. We will not shut down the job server until all users have migrated. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0lIyIACgkQIXd7fCuc5vLYxQCgkpBdrvdYiCBDe/4QR2Xx4tDU K/AAn1LeTo77idvq4jYB72HbNeqrSwor =je9C -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] cron
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Since the switch to Solaris a few people have complained about the Sun 'cron' not being very good. As an alternative to cron, we have now installed 'cronie', which is the RedHat Linux version of cron. This is based on Vixie cron and offers an almost identical feature set to the old cron on nightshade. To avoid a conflict with the system cron, the cronie crontab command is called 'cronie'. To convert your current crontab to a cronietab[0], you can use a command like this: % crontab -l crontab # list existing crontab to file % cronie crontab # install same as a cronietab % crontab -r # remove old crontab You should probably use either crontab or cronie, but not both, since that could be confusing. - river. [0] Yes, cronietab is a fairly stupid name. Sorry. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0lXhoACgkQIXd7fCuc5vKuEACdHkfvuZeTdT8MmMmNpgY1O/t6 7PwAn1tk+uJ9czNa+MBWQWP4vndzN+Et =FIKF -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Maintenance
emijrp: 2011/1/3 River Tarnell river.tarn...@wikimedia.de $ crontab $HOME/crontab.before_nightshade_reinstall I get errors http://toolserver.org/~emijrp/cronerrors.txt You may want to review https://wiki.toolserver.org/view/Using_Solaris#cron before installing the new crontab. Please read the page I linked to. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Compressing stats files better (was; Re: /mnt/user-store is full)
emijrp: We can compress to xz while the new disks arrive. I read that it is about 24 TB, so, we can revert to gzip in the future. I suggested xz pretty much as an emergency fix; we don't want to delete the files, but they do take up a lot of space. I don't mind switching back to gzip when we have more space; in fact, I already suggested this to schutz in private. However, if xz works okay (and it should, since the interface is identical to gzip) we may as well stay with it. If it turns out to be too slow, we could consider 7z or something else (rzip, bzip2, ..., or maybe even just gzip -9). - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Compressing stats files better (was; Re: /mnt/user-store is full)
emijrp: We can compress to xz while the new disks arrive. I read that it is about 24 TB, so, we can revert to gzip in the future. PS: Yes, we are installing 24TB of disks (12x 2TB disks), but this is raw space. To begin with, a 2TB disk is only 1,862 GB of real space. Assuming we configure the disks as RAID 50 with two 6-disk legs, that gives (1862*(6-1))*2 = 18,620 GB (18.2TB) usable space. We will reserve some of this for internal use (such as backups), so the total amount available to users will be less than that. When budgeting for this upgrade, we assumed 5TB would be used for user-store. In reality, there should be a lot more than this (we originally planned to use 1TB disks); but it won't be a full 24TB. - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Maintenance
Hi, The previous announced maintenance has been completed successfully. In case of problems, open an issue in the TS project in JIRA. For problems relating to the conversion of nightshade to Solaris (e.g. missing software), open an issue in the TS project and include [solaris] at the start of the subject line. Due to the disruptive nature of this maintenance, such requests will be given priority. (However, I'll be away for at least the next 12 hours, so I won't be able to resolve them immediately.) - river. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Maintenance
Note: crontabs from nightshade have been saved as $HOME/crontab.before_nightshade_reinstall. You should review this for any necessary changes, then install it: $ crontab $HOME/crontab.before_nightshade_reinstall Until you do this, your crontab will not run. You may want to review https://wiki.toolserver.org/view/Using_Solaris#cron before installing the new crontab. - river/ ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] General maintenance notice: January 3rd, 2011
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On the morning (UTC) of January 3rd we will performance general maintenance[0] on all servers. Services will be affected as follows: Service | Expected impact --+-- Login server: nightshade | See below. Login/web servers | Some software unavailable for 30 minutes. All other services| No impact. Start time: Monday, 3rd January, 12AM UTC http://www.timeanddate.com/worldclock/fixedtime.html?day=3month=1year=2011hour=0min=0sec=0p1=0 End time: Monday, 3rd JAnuary, 3AM UTC (estimated) http://www.timeanddate.com/worldclock/fixedtime.html?day=3month=1year=2011hour=0min=0sec=0p1=0 Details: We will perform general software upgrades. A list of software to be upgraded can be found at: https://wiki.toolserver.org/view/Admin:Pending_maintenance_tasks Some software may be unavailable or function incorrectly during the upgrade process, which we estimate will take under 30 minutes. -- Python 2.6 (/usr/bin/python2.6, /opt/ts/python/2.6/bin/python) will be removed during the maintenance. This was announced in the previous maintenance and no problems have been reported with Python 2.7, which is currently the default Python. -- The default locale on the Solaris login servers will change to en_US.UTF-8. This was already the case for all services except cron. -- We will reinstall nightshade, the Linux login server, as Solaris. This was announced in September and again in November. No major issues have been reported which would block the migration. For more information, see http://lists.wikimedia.org/pipermail/toolserver-announce/2010-September/000351.html and http://lists.wikimedia.org/pipermail/toolserver-announce/2010-November/000369.html - river. [0] https://wiki.toolserver.org/view/Maintenance_schedule -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk0Z+m8ACgkQIXd7fCuc5vK7ZgCfRUJNpjo7AXpoh/r48B92YPkb BvUAn27En7d+5q6qPwNWBBSg6kJLzv7J =xTT4 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] /mnt/user-store is full
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Aryeh Gregor: I'm looking at this, but I have no brighter idea than just running du on it, which is taking a very long time. I wrote a tool called summdisk for this, which produces reports on per-user disk usage: http://lists.wikimedia.org/pipermail/toolserver-announce/2010-September/000343.html Unfortunately it still takes a very long time to run. According to df, there are 136,110,408 inodes used on the volume[0]; I wonder if people who currently create large numbers of small files could save some accounting space by aggregating them into larger blocks, like OSM's meta-tiles. - river. [0] of which ~127m or 93% are used by a single user -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (NetBSD) iEYEARECAAYFAk0YgfUACgkQIXd7fCuc5vIv8ACguhvsRsc+lRi6hGOaEpWR/Xfg cxYAn2sjtEkeeG+uoWrvTvH/5fQQ4U54 =WWcS -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Reboot of amaranth
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, At 19:20 UTC Wikimedia accidentally rebooted amaranth (the Toolserver system in the US) during maintenance. This caused an outage of JIRA, MediaWiki, FishEye and MySQL database replication for around 5 minutes. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (NetBSD) iEYEARECAAYFAk0JFhIACgkQIXd7fCuc5vLatQCdHgJjQYDJxDQFOs55CNbDHeoS HwEAnj0iQldsfA5kwkRjafrLHu9+wSXL =lDUg -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] mysql quries killing script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mashiah Davidson: Am I right that I need to check s3 replag as well for this particular case but do not have to check replag on s2, s5 and s1? Yes. The server sets are: s1; s3/s6; s2/s5. In addition commons (s4) is present on all servers. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0DekoACgkQIXd7fCuc5vJTbgCgvN2cMXIBLp/3SYdqN4KqY2xk rF8An3ex2LbWKYHd4Ave4cJ2Zg3Yv6KX =zO0U -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] mysql quries killing script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mashiah Davidson: ERROR 1317 (70100) at line 6874: Query execution was interrupted last replicated timestamp: 20101207214400 replag: 00:00:01 Could anyone explain whether it was possible that a query (even a long running one) has been killed when replag was so good? Did you check the replication lag on all clusters on the server your query was running on, or only one? The query killer (correctly) ignores the cluster the database cluster the query is running on, since queries on one database can still affect replication of another cluster. (If your query was running on s1, this doesn't apply.) - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0CAiwACgkQIXd7fCuc5vJXdQCfSuFGM/OeC0vY11Jwf0N+S4GU pXIAnj3gutXVGax2pE0p7zFPl19NhOsx =kc7P -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Two beginner questions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 MZMcBride: Gahhh, this list. Nobody suggested just using Python's Twisted? Someone is suggesting it: you. That's pretty much the point of the list; there's more than one person on it. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0CArMACgkQIXd7fCuc5vIz7ACgpAbHWADakLzMbQfIjW2rcA0A NpkAn33w1ebQEAIzKKGDcii+W9jXKgZy =NBwd -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Two beginner questions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Михајло Анђелковић: I was returning the server whatever it was sending to me as a ping. This is how it worked like two years ago. Has something changed? This is the correct way to do it, but many IRC client implementations are lazy and are written by people who guess the protocol, and end up with an incomplete understanding, or simple don't care enough to implement it correctly since the incorrect solution works in most cases. (As someone else pointed out, many IRC servers (especially hybrid-derived ones) use a ping cookie on connect; it was originally intended to defeat source address spoofing, but more recently has been used against other attacks. If you don't reply correctly to a ping from one of these servers, you won't be able to connect.) - river. PS: I cringe every time I see someone parsing IRC lines with things like strncmp(line, PRIVMSG , 8) or strstr(line, :). The IRC protocol is very simple, and tokenising it properly is really not that difficult. (Every argument is separated by a space; if the first byte of the argument is :, remove it and stop splitting.) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0CA+0ACgkQIXd7fCuc5vIveQCfcbOkmQmGRxNCEZRLWuzLg51B 1w0Anil3POujv+MvallFnjAesOGYsoEL =mL0N -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] mysql quries killing script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mashiah Davidson: The script was working with just one DB cluster (s3/s6), so I've checked just it's replication lag. There isn't an s3/s6 replication lag. There's an s3 replication lag and an s6 replication lag. If your query was running on an s3 database, and you checked the replication lag there, you might have missed the fact that s6, on the same server, was lagged. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0CBqEACgkQIXd7fCuc5vKynwCfcZ0wtIsIZgmBdeLmqS/CaYnX 7aMAoLpFOkTq+VxqJgI3JjJk5ZtvFuQ+ =/GFQ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Two beginner questions
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Sumurai8 (DD):
text = IRC.recv(1024)
msgs = text.split('\n')
This seems to have a bug: if there's more than 1024 bytes waiting, you could
receive only part of the final message; so you will truncate that message, and
the next recv will receive the other half (which will then be effectively
junk).
- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)
iEYEARECAAYFAk0A/0QACgkQIXd7fCuc5vKX8QCeKN77J7YXVJaO5utUVMyxCC5a
ubsAnR/+E/8WtjZuD1Qrc78S5v68ZQ5/
=z4ru
-END PGP SIGNATURE-
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] General maintenance notice: December 6th
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This maintenance was mostly completed. Upgrade of web applications (JIRA/MediaWiki) was postponed and will be finished tomorrow. In case of problems, please file an issue in JIRA. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz8u+wACgkQIXd7fCuc5vIt6gCeIqoR+RgrEwumRr9xUOlb5Eo1 RNAAniOElwa5Ugt55olUUaF3ywTrg9bk =TX7+ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s2/s5
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The suspected faulty part has been replaced and MySQL is now running again. We will continue to monitor the system over the weekend. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz499EACgkQIXd7fCuc5vLV7wCgpqVl73Grww9JCDRuGfpusWNs hWcAniCSQtXESiQ3mPJ63TR/83xnYTRy =4eEz -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s2/s5
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bryan Tong Minh: It's amazing how fast you guys fixed this issue. Well, mostly I just sit at home and call Sun, and they send someone to fix the problem. So the speed of the fix is directly proportional to how much we pay Sun ;-) - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz4/w0ACgkQIXd7fCuc5vK51gCeN4qs90epFt+FSnZGgoLf0rfM ZmUAnAtz5S61Z7bUeEg6Dx52XuETaw9Y =tnnS -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s2/s5
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: The s2/s5 server is currently experiencing some hardware issues. As we currently only have a single server for s2/s5, this may result in outages for these clusters. Hi, We have identified the cause of the fault and do not expect any further failures. However, we will need to shut down the system either today or later this week to replace the faulty part, which will cause under 30 minutes outage for s2/s5. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz3l+QACgkQIXd7fCuc5vIwZwCfR2MLssyLBpG95IFQjjwavWB0 tOMAmwakPUb2idBQfrJpUaf9JGRJkIfh =jgeP -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] Hardware failure affecting s2/s5
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: However, we will need to shut down the system either today or later this week to replace the faulty part, which will cause under 30 minutes outage for s2/s5. This maintenance has been scheduled for 1030h UTC[0] tomorrow (3rd Dec). - river. [0] http://www.timeanddate.com/worldclock/fixedtime.html?day=3month=12year=2010hour=10min=30sec=0p1=0 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz3qosACgkQIXd7fCuc5vJG0gCggJFnPowBnyujHBnzdrGtnnLD mbEAn032FO/GcMVq6A+FSM/KKojs0d1b =KQin -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] nightshade will be converted to Solaris on January 3, 2011
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jacobi Carter: Will my statically compiled Go code run on Solaris like it did on Linux? No, compiled software needs to be recompiled for Solaris. There is currently no Go compiler for Solaris, but if you need one, we can look at porting either of the existing compilers - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz4besACgkQIXd7fCuc5vIxMwCeKZjp3z5uSfAUNBk1IyJs+MSA 7G8AoKUQ5ZNRxBQYgGAOyMZw0iPzJg94 =FTKC -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Software update schedule
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, After this month's maintenance, which includes quite a few software updates/changes, I'd like to come up with a coherent strategy for updating software. The aim is to balance the inconvenience of updates (which can require changes from users) with having reasonably up-to-date software. This will apply to all programs, libraries and modules (Perl/Python/...) in /opt/ts on the Solaris servers. I would propose the following schedule: * New installs will be done on demand * Minor upgrades (e.g. 1.0.0 to 1.0.1) will be done each month * Major upgrade (e.g. 1.0.0 to 1.1.0 or 1.2.0), which might require changes from users, will be done quarterly, in January, April, July and October. In addition, when we upgrade a shared library (e.g. libjpeg.so.7 - libjpeg.so.8), or a large application like Python (2.x - 2.y), if possible, we will leave the old version installed for one maintenance cycle (3 months) to give users time to fix or recompile local software. Please comment if you think this won't work for you, or if you have a better idea. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz1ObAACgkQIXd7fCuc5vJMYgCgkjjihBfiQx0IR/o48zDuaYjw 72IAoLw2A2c0rZt0pC+1ZDPzSZzO6uGD =/Dp6 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Are months shorter in 2011?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Nakor: I renewed my account today for 6 months and the new expiration date is May 17 2011. I do not care the two weeks difference but just wanted to check if it was a bug in the renewal tool. It actually renews your account for 168 days (24 weeks). This is based on a 4-week month, which obviously isn't quite the same as reality. I will change it to use 182 days, which is a bit closer. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz1SB0ACgkQIXd7fCuc5vIRAgCfav4mPtnGHD84BPonkEtIKVID 2mcAnj/c3fLemlTNkk+cAJlMGfGd6DEP =6oUF -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Extracting basic revision data
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Михајло Анђелковић: WHERE rev_id 0 AND rev_id [...] AND rev_deleted = 0; Please check that MySQL plans this correctly (using the rev_id index). If this is generally allowed to do, my question is how large chunks of data can I take at once, and how long should be waited between two takes? A simple fetch like this doesn't usually take many resources. You can probably pick any random number for chunking. If you run each chunk under SGE[0] as a separate job, and request an sqlprocs resource, SGE will handle scheduling for you so there's no need to worry about how long to wait between runs. - river. [0] https://wiki.toolserver.org/view/Job_scheduling -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkzzsQgACgkQIXd7fCuc5vI7ZACbBkmQAFXTSBEDYm+I8xf+Ci4X LFQAnjOPp1zFUQ/Njs90nQ4us4IZswZE =mPqu -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] General maintenance notice: December 6th
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi,
On the morning (UTC) of December 6th we will perform general maintenance[0] on
all servers. Services will be affected as follows:
Service | Expected impact
-+-
Entire platform | As described in maintenance schedule[0]
JIRA, MediaWiki, FishEye | Under 30 minutes outage for each service
| during upgrade
Start time: Monday, 6th December, 12AM UTC
End time: Monday, 6th December, 8AM UTC (estimated)
Details:
This is a schedule general maintenance, which we use for various non-critical
tasks. The expected outages are as described in the maintenance schedule[0].
Most of the changes for this maintenance are in the local TS software, /opt/ts;
all software will be upgraded to the latest version, and some minor changes
will be made. The full list of upgrades, including Perl modules, is available
here:
https://wiki.toolserver.org/view/Admin:Pending_maintenance_tasks
The following changes will also be made:
* Mono will now install directly in /opt/ts instead of /opt/ts/mono/2.0. If
you call mono without an absolute path, this will not affect you. If you
currently call /opt/ts/mono/2.0/bin/mono, you should change this to remove
the full path before the maintenance.
* The preferred OpenSSL is now /opt/ts/bin/{amd64,}/openssl, which is OpenSSL
1.0.0b instead of /usr/sfw/bin/{amd64,}/openssl (0.9.7d). This should not
affect users, but if you currently call the version in /usr/sfw with a full
path, you may wish to remove the path so you automatically use our version,
which is better.
We will additionally make some changes to how the software is compiled; if you
have compiled your own C or C++ programs, this will affect you, and you should
read the section Changes to ts-specs environment below. If you do not have
any locally-compiled software, this change will not affect you.
During the maintenance, some software may not work correctly (e.g. programs or
libraries not found).
--
JIRA, FishEye, MediaWiki and phpMyAdmin will also be upgraded.
--
The default Python version will change from 2.6 to 2.7. If you currently use
/usr/bin/python, this change will happen for you automatically. If you use
/usr/bin/python2.6 explicitly, you will need to change to /usr/bin/python2.7 to
use the new Python.
If you have programs which don't work under Python 2.7, you should a) report
this in JIRA, and b) switch to /usr/bin/python2.6 before the maintenance. If
there are no problems with Python 2.7, we will remove Python 2.6 from the
system during the next maintenance (January 2011).
We will patch Python 2.7 to revert the fix for bug 1054943[1], which introduced
a regression affecting Unicode normalisation[2][3].
--
The default gcc will become GCC 4.5.1, rather than 3.4.3. This may affect
you if you build locally-installed Perl modules, especially if these modules
use C++ code. This is described in more detail below.
--
Changes to ts-specs environment
===
This section only applies to users with locally-compiled C or C++ software.
The new version of ts-specs (/opt/ts) installed during the maintenance has
switched the default compiler from Sun Studio to GCC 4.5.1. This is described
in more detail at [4]. In brief:
* You should change from your current compiler (Studio, GCC 3.4.3 or GCC 4.4)
to GCC 4.5.1, /opt/ts/bin/gcc.
* If you recompile any Studio- or GCC 3.4.3-compiled C++ code with the new
compiler, you need to recompile all of it, because the ABIs are not
compatible.
* GCC 3.4.3 will no longer be installed.
* If you use Studio-compiled versions of C++ libraries in
/opt/ts/lib/version/, you should change to the GCC version in
/opt/ts/lib/version-gcc/.
The following libraries require special handling:
* Studio-compiled versions of MySQL++ and VIPS were previously installed in
/opt/ts/lib. If you use these libraries, you should switch to the version
that we will install in /opt/ts/lib (where lib is mysqlpp or vips).
The Studio-compiled version will remain available for now.
* Studio-compiled versions of ImageMagick, sigc++ and cairomm are currently
installed in /opt/ts. They will be replaced with GCC-compiled versions
in the same path, and Studio-compiled versions will not be available.
If this adversely affects you (because you use these libraries and need time
to migrate), you should let us know before the maintenance.
[0] https://wiki.toolserver.org/view/Maintenance_schedule
[1] http://bugs.python.org/issue1054943
[2] http://bugs.python.org/issue10254
[3] http://lists.wikimedia.org/pipermail/toolserver-l/2010-November/003633.html
[4]
http://lists.wikimedia.org/pipermail/toolserver-announce/2010-November/000367.html
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)
Re: [Toolserver-l] General maintenance notice: December 6th
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Marco Schuster: please use a service like http://www.timeanddate.com/worldclock/ in the future when specifying dates/times. Sorry, I don't understand what you're asking for here. Can you give an example? - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz0WxIACgkQIXd7fCuc5vJkrgCdHvSRaFbU7axY0bm2tUYZtoPc f4IAoKjqVE+OuzedJbCKUDuJ9OtFNZfF =aZdn -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Problem with matplolib / pylab
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robin Krahl: ImportError: ld.so.1: python: fatal: relocation error: file /opt/ts/python/2.6/lib/python2.6/site-packages/matplotlib/_path.so: symbol __1cDstdMbasic_string4Ccn0ALchar_traits4Cc__n0AJallocator4Cc___J__nullref_: referenced symbol not found This should be fixed now. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkzyS98ACgkQIXd7fCuc5vLFwQCeL0y+WqdPz1uAjfzuyn+0J6Ox qdcAnijpm4C1u8YLmWRcx0s4ANYttt2h =4HrO -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Changes to software libraries in /opt/ts
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mauro Girotto: 2010/11/14 River Tarnell river.tarn...@wikimedia.de: I'm trying to switch from CC to gcc. My app uses mysqlpp library. Does /opt/ts/lib contain the libraries of Sun Studio? If I compile with -L /opt/ts/lib -lmysqlpp I've a linking error: Undefined symbol mysqlpp::Row::at(unsigned int) const mysqlpp::UseQueryResult::fetch_row() mysqlpp::Query::error() const mysqlpp::Connection::query(char const*) ... You're right, we neglected to provide a GCC version of mysqlpp. I will install one during the ts-specs update and you will be able to link with that instead. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkzyZSQACgkQIXd7fCuc5vLg2ACfeKwHQTAP1WN6uNshP+iSpTN0 fzoAn3dmj3dJQKgTJM0PlWnaMvnXFjsO =Y3w3 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] alternative way to get wikipedia dump while server is down
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 (Removing wikitech-l cc since I'm not subscribed there.) Platonides: Roan Kattouw wrote: 2010/11/26 Bryan Tong Minh bryan.tongm...@gmail.com: Somehow I think that publishing an entire dump violates the do not publish significant parts of an article rule. Surely the toolserver admins could be asked to consider waiving that in this case considering the public nature of the dumps and the downtime situation with download.wm.o It's not that toolserver admins are excentric adding such rule, but an issue of WM-DE liability if such information is published. This is correct -- the rule applies to any article text, even articles already available on Wikimedia foundation-run websites. (Otherwise, there would be no point having the rule at all, since *all* text on the Toolserver is also available on the site itself.) - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkzwVCMACgkQIXd7fCuc5vIx7QCfVRkyskknRHUkhzhraaJOV+z3 PwwAn0dVwRv3hGcjWpxTvD4PSQn7fJFQ =MfZC -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] php scripts timing out after 2 minutes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ryan Kaldari: My PHP scripts for the contests tool seem to be consistently timing out after 2 minutes now. I have max_execution_time set to 7 minutes in both the scripts themselves and in .htaccess. The contests tool is pretty much unusable with a 2 minute timeout since average batch upload time for a team is about 3-4 minutes. Can you still reproduce this problem? - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkzwWaAACgkQIXd7fCuc5vKX2wCcCG1U8wNPVcHcC9d9aEt82/2j C3wAoKLCEqY7gMLekonupB1ypJvCBRKf =z/kG -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] php scripts timing out after 2 minutes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ryan Kaldari: My PHP scripts for the contests tool seem to be consistently timing out after 2 minutes now. I don't see anything in the sever configuration that would cause this. I've raised an issue with Zeus support. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkzroh8ACgkQIXd7fCuc5vKOTQCgrVJ3xiz0y/a7inOrEWLwKmhm DqgAn11YWKkP9lAnJVigyMxxA4JM1Fvk =QtFK -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Default Python will change to 2.7
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, During the general maintenance on Dec 6th, we will change the default Python version (/usr/bin/python) on the Solaris user servers from 2.6 to 2.7. You may wish to test your tools with /usr/bin/python2.7 before then. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkzqREgACgkQIXd7fCuc5vIhFQCgpX20z0B9xHikuwl+yiEUDzFH WjYAn1wqm21wZjP1uQhsEO7RkxlTyE/N =CqUE -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Newbie question: definition of a heavy job for toolserver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex Brollo: My question is: how many pywikipedia readings/editing per hour are a heavy toolserver job? We don't usually measure resource use in pywikipedia readings/editing per hour ;-) The most common indicators of resource use are CPU time and memory. There's also no clear point at which a job becomes heavy -- not least because heavy is a vague description and could mean different things to different people. It would be easier to answer your question if you could provide some context... - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkzjmyQACgkQIXd7fCuc5vLa1gCgvkh/jEIMXULyiIZgEFREDu+5 WgMAniK+5D3fPdR424DZT6k9zBjrsx0d =bD4i -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Fast local storage
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, A feature we may provide to users in the future is servers with fast local (not NFS) storage for use by tools. This would be something like user-store, except per-server. This storage would be redundant (RAID), but not backed up. If anyone feels like they may make use of this, it would be useful to know exactly how you plan to use it, e.g. how much storage you would need and for how long, as well as the sort of tasks it would be used for. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkzkYpAACgkQIXd7fCuc5vJDygCghiTRX+/t62T4aAudXo0MoX1F hCcAn22PuG2jWQCUnirte70sUwx2A/j4 =qPZo -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Fast local storage
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Marco Schuster: I can imagine a use of this for everything that uses the static or xml dumps, as well as unpacking or packing large files. Do you have any numbers for a particular use case? - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkzkZWYACgkQIXd7fCuc5vKL/QCgudneKt9mjj7NST7Gw/n0Ciki pZMAn3tG5Q2sbJunNfQReY5JvnB+3XZK =RBOK -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Toolserver-announce] SGE on web servers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The webservers (wolfsbane and ortelius) have been added as SGE submit hosts, Since the webservers have a lot of idle CPU (about 90%), I've now added them as SGE execution hosts as well, so SGE jobs will run on these systems. We'll monitor to make sure this doesn't create too much load. Note that running ordinary tools on these systems still isn't allowed. If you want to take advantage of the extra capacity, you would need to use SGE. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkzh9j4ACgkQIXd7fCuc5vIC9wCdGPDSFh3BXfgk0hkII6uBVk52 ECwAnifrbqQaYp9Z+KICs9hgASX9bx+9 =Utd1 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] SGE can now be used for *all* tools
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, In the past, our policy on SGE (batch job scheduling) was that it should only be used for jobs which run once then exit, and that it should not be used for jobs which run continuously. This policy has now changed, and SGE can be used for all jobs, including long-running jobs. This effectively obsoletes Phoenix and similar tools, since SGE with cronsub provides the same functionality. Because SGE makes it easier for us to scale the Toolserver cluster, as well as making your tools run more efficiently (and therefore quicker), we recommend that all tools be converted to SGE where possible. The wiki page now has a quick-start section explaining how to do this: https://wiki.toolserver.org/view/Batch_job_scheduling#Quick_start (While SGE has a lot of features and can be complicated, we've tried to make it easy to use for the typical cases; if something is not clear, please let us know.) - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkziD9YACgkQIXd7fCuc5vLmewCeLTy/5zyomUajAttXfPoG7j+e vToAnRD5yVeL5iccv/SfzIgGicHSoAS0 =fV93 -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] SGE can now be used for *all* tools
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Alex Brollo: Thanks. A question. I'm a new toolserver user, presently working at a long list of personal Hello world tests. Next scheduled Hello world was cron test. Is your suggestion to step over cron, and to try to learn SGE from the beginning? If you're new to Unix, I would suggesting learning cron first, then adding SGE later. You'll need to use cron to run your SGE jobs anyway. Converting an existing cron job to SGE is quite simple: https://wiki.toolserver.org/view/Batch_job_scheduling#Running_jobs_from_cron - river. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.16 (FreeBSD) iEYEARECAAYFAkziHvAACgkQIXd7fCuc5vLs8gCgnWvjXTp7dZvVTdUhAbPGNHw/ 07EAn1AymtluBv+TJs9wuqetTkreWST8 =WHM/ -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
