Re: [Toolserver-l] General maintenance notice: December 6th

[Marco Schuster]

Hi,

when you click on World Clock => Fixed Time World Clock, you can enter
a date/time combo.
Given the start date of 06/12/2010 00:00:00 UTC, I'd prefer
http://www.timeanddate.com/worldclock/fixedtime.html?day=6&month=12&year=2010&hour=0&min=0&sec=0&p1=0
to be linked in the email.

This way an user from Berlin can immediately see the maintenance
starts at Monday 01:00 in the morning and an user from Toronto/Canada
can see the maintenance will affect him beginning at 19:00 Sunday
local time.
No stupid conversion issues for the people who don't get the thing
behind time zones (like me, for the hell of it - 12h format is such a
mess) :)

Marco

On Tue, Nov 30, 2010 at 3:01 AM, River Tarnell
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Marco Schuster:
>> please use a service like http://www.timeanddate.com/worldclock/ in
>> the future when specifying dates/times.
>
> Sorry, I don't understand what you're asking for here.  Can you give an
> example?
>
>        - river.
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.11 (FreeBSD)
>
> iEYEARECAAYFAkz0WxIACgkQIXd7fCuc5vJkrgCdHvSRaFbU7axY0bm2tUYZtoPc
> f4IAoKjqVE+OuzedJbCKUDuJ9OtFNZfF
> =aZdn
> -END PGP SIGNATURE-
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>



-- 
VMSoft GbR
Nabburger Str. 15
81737 München
Geschäftsführer: Marco Schuster, Volker Hemmert
http://vmsoft-gbr.de

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: December 6th

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Marco Schuster:
> please use a service like http://www.timeanddate.com/worldclock/ in
> the future when specifying dates/times.

Sorry, I don't understand what you're asking for here.  Can you give an 
example?

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAkz0WxIACgkQIXd7fCuc5vJkrgCdHvSRaFbU7axY0bm2tUYZtoPc
f4IAoKjqVE+OuzedJbCKUDuJ9OtFNZfF
=aZdn
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: December 6th

[Marco Schuster]

Hi River,

please use a service like http://www.timeanddate.com/worldclock/ in
the future when specifying dates/times. This way you can use the 12h
format you're used to, but the non-English people who use the 24h
format or do not live in European time zones can exactly see when the
maintenance affects them.

Marco

On Tue, Nov 30, 2010 at 1:10 AM, River Tarnell
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Aryeh Gregor:
>> On Mon, Nov 29, 2010 at 11:27 AM, River Tarnell
>>  wrote:
>> > Start time: Monday, 6th December, 12AM UTC
>> > End time: Monday, 6th December, 8AM UTC (estimated)
>
>> Are these reversed or what?
>
> No.  12AM (h) is before 8AM (0800h).
>
>        - river.
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.11 (FreeBSD)
>
> iEYEARECAAYFAkz0QP8ACgkQIXd7fCuc5vJzJgCfZXicftOZVYlS7djkPbwq1mpf
> kQIAmwXBgiXexBDiAxGxKI07TRMilHNf
> =QdOM
> -END PGP SIGNATURE-
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>



-- 
VMSoft GbR
Nabburger Str. 15
81737 München
Geschäftsführer: Marco Schuster, Volker Hemmert
http://vmsoft-gbr.de

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: December 6th

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Aryeh Gregor:
> On Mon, Nov 29, 2010 at 11:27 AM, River Tarnell
>  wrote:
> > Start time: Monday, 6th December, 12AM UTC
> > End time: Monday, 6th December, 8AM UTC (estimated)
 
> Are these reversed or what?

No.  12AM (h) is before 8AM (0800h).

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAkz0QP8ACgkQIXd7fCuc5vJzJgCfZXicftOZVYlS7djkPbwq1mpf
kQIAmwXBgiXexBDiAxGxKI07TRMilHNf
=QdOM
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: December 6th

[MZMcBride]

Aryeh Gregor wrote:
> On Mon, Nov 29, 2010 at 11:27 AM, River Tarnell
>  wrote:
>> Start time: Monday, 6th December, 12AM UTC
>> End time: Monday, 6th December, 8AM UTC (estimated)
> 
> Are these reversed or what?

I don't follow. A day starts at 12:00 AM and goes until 11:59 PM. River's
post seems fine to me, though a lot of people use 00:01 UTC (as opposed to
00:00 UTC or 12 AM UTC) in order to avoid exactly this type of confusion.

MZMcBride



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: December 6th

[Krinkle]

Yeah, I thought the same at first but I had this confusion before.

It goes like this for comparison to european / 24h times :

24 notation:
00:00, 08:00, 11:00, 12:00, 20:00, 23:00, 24:00

Are the same as these in that order:
12 AM, 8 AM, 11 AM, 12 PM, 8 PM, 11 PM, 12 AM

--
Krinkle

Op 30 nov 2010, om 01:22 heeft Kalan het volgende geschreven:

> On Tue, Nov 30, 2010 at 03:04, Aryeh Gregor
>  wrote:
>>> Start time: Monday, 6th December, 12AM UTC
>>> End time: Monday, 6th December, 8AM UTC (estimated)
>> Are these reversed or what?
>
> They read 2010-12-06 00:00 and 2010-12-06 08:00 respectively, no  
> reversing.
>
> 12-hour clock, and specifically its AM/PM notation, is so stupid and
> confusing. Especially for the countries preferring to stay away from
> it.
>
> — Kalan
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: December 6th

[Kalan]

On Tue, Nov 30, 2010 at 03:04, Aryeh Gregor
 wrote:
>> Start time: Monday, 6th December, 12AM UTC
>> End time: Monday, 6th December, 8AM UTC (estimated)
> Are these reversed or what?

They read 2010-12-06 00:00 and 2010-12-06 08:00 respectively, no reversing.

12-hour clock, and specifically its AM/PM notation, is so stupid and
confusing. Especially for the countries preferring to stay away from
it.

— Kalan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] General maintenance notice: December 6th

[Aryeh Gregor]

On Mon, Nov 29, 2010 at 11:27 AM, River Tarnell
 wrote:
> Start time: Monday, 6th December, 12AM UTC
> End time: Monday, 6th December, 8AM UTC (estimated)

Are these reversed or what?

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[Михајло Анђелковић]

Hm, that is very correct. The data I've got do not have this info.

But I won't run such a query again soon, since this still does the
job: for now I only want to acknowledge when somebody has left sr.wp
and to book the reason by reviewing the talk and other relevant pages
from that time.

Thank you both for the tip.

M

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Михајло Анђелковић:
> Namespaces are easily determined from the page prefix, I am not
> bothered if there are any anomalies out there (i.e. page starting with
> "User talk:" being in NS 0)

There are no page namespace prefixes in the databases.  IOW, "Foo" and 
"Talk:Foo" both have page_title="Foo".

I can't believe that's what you want, since the output will be useless.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAkz0C1AACgkQIXd7fCuc5vKVDQCfUXpleT++0/KTGPCB4Iv5fuh8
MVUAoIOjDKBPbW5NhRssqr+luRNrx13q
=9aVE
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[Михајло Анђелковић]

Thank you, guys, I've already taken what I needed.

Namespaces are easily determined from the page prefix, I am not
bothered if there are any anomalies out there (i.e. page starting with
"User talk:" being in NS 0) and the query is lighter in case ns isn't
being pulled out from the DB. In overall, it was taking about 17
seconds to write down the data about 1M revisions.

Setting span for rev_ids was only to take the data in chunks, I didn't
have to specify them if I wanted to take them all at once. But hey,
such chunks are even easier to sort by rev_id.

M

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Monthly pageviews

[MZMcBride]

Krinkle wrote:
> Op 15 nov 2010, om 19:22 heeft Magnus Manske het volgende geschreven:
>  
>> I know there are lots'o'files for daily (hourly?) pageview stats on
>> the toolserver.
> 
> Where are these text files actually ?

/mnt/user-store/stats/

MZMcBride



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[MZMcBride]

Михајло Анђелковић wrote:
> I would ask for allowance to run a request that can be resource
> consuming if not properly scaled:
> 
> SELECT page.page_title as title, rev_user_text as user, rev_timestamp
> as timestamp, rev_len as len FROM revision JOIN page ON page.page_id =
> rev_page WHERE rev_id > 0 AND rev_id < [...] AND rev_deleted = 0;
> 
> This is intended to extract basic data about all publicly visible
> revisions from 1 to [...]. Info about each revision would be a 4-tuple
> title/user name/time/length. I need this data to start generating a
> timeline of editing of srwiki, so it is intended to be run only once
> for each revision.
> 
> If this is generally allowed to do, my question is how large chunks of
> data can I take at once, and how long should be waited between two
> takes?

srwiki_p isn't very large (3665333 revisions and 413987 pages), so I
personally wouldn't worry about performance very much at all. If you were
going to run this query on enwiki_p or another larger database, it might be
more of a concern. Run the queries that you need to run.

The "Queries" page on the Toolserver wiki might be helpful to you.[1]

Looking at your query, you should pull page.page_namespace or specify
page_namespace = 0. Pulling only page.page_title without specifying a
namespace will output useless results. I'm also unclear why you'd need to
specify rev_id > 0, though you might have your reasons for doing so.

Your Toolserver account has a quota (viewable with 'quota -v') that you
might hit if you're outputting a lot of data to disk. You can always use
/mnt/user-store/ or file a ticket in JIRA if you need an increased quota.

MZMcBride

[1] https://wiki.toolserver.org/view/Queries



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] General maintenance notice: December 6th

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

On the morning (UTC) of December 6th we will perform general maintenance[0] on 
all servers.  Services will be affected as follows:

Service   | Expected impact
 -+-
Entire platform   | As described in maintenance schedule[0]
JIRA, MediaWiki, FishEye  | Under 30 minutes outage for each service
  |   during upgrade

Start time: Monday, 6th December, 12AM UTC
End time: Monday, 6th December, 8AM UTC (estimated)

Details:

This is a schedule general maintenance, which we use for various non-critical
tasks.  The expected outages are as described in the maintenance schedule[0].

Most of the changes for this maintenance are in the local TS software, /opt/ts;
all software will be upgraded to the latest version, and some minor changes 
will be made.  The full list of upgrades, including Perl modules, is available 
here:
  

The following changes will also be made:

* Mono will now install directly in /opt/ts instead of /opt/ts/mono/2.0.  If 
  you call "mono" without an absolute path, this will not affect you.  If you
  currently call "/opt/ts/mono/2.0/bin/mono", you should change this to remove
  the full path before the maintenance.

* The preferred OpenSSL is now /opt/ts/bin/{amd64,}/openssl, which is OpenSSL 
  1.0.0b instead of /usr/sfw/bin/{amd64,}/openssl (0.9.7d).  This should not
  affect users, but if you currently call the version in /usr/sfw with a full
  path, you may wish to remove the path so you automatically use our version,
  which is better.

We will additionally make some changes to how the software is compiled; if you 
have compiled your own C or C++ programs, this will affect you, and you should 
read the section "Changes to ts-specs environment" below.  If you do not have 
any locally-compiled software, this change will not affect you.

During the maintenance, some software may not work correctly (e.g. programs or 
libraries not found).

 --

JIRA, FishEye, MediaWiki and phpMyAdmin will also be upgraded.

 --

The default Python version will change from 2.6 to 2.7.  If you currently use 
/usr/bin/python, this change will happen for you automatically.  If you use
/usr/bin/python2.6 explicitly, you will need to change to /usr/bin/python2.7 to 
use the new Python.

If you have programs which don't work under Python 2.7, you should a) report 
this in JIRA, and b) switch to /usr/bin/python2.6 before the maintenance.  If 
there are no problems with Python 2.7, we will remove Python 2.6 from the 
system during the next maintenance (January 2011).

We will patch Python 2.7 to revert the fix for bug 1054943[1], which introduced 
a regression affecting Unicode normalisation[2][3].

 --

The default "gcc" will become GCC 4.5.1, rather than 3.4.3.  This may affect 
you if you build locally-installed Perl modules, especially if these modules 
use C++ code.  This is described in more detail below.

 --

Changes to ts-specs environment
===

This section only applies to users with locally-compiled C or C++ software.

The new version of ts-specs (/opt/ts) installed during the maintenance has 
switched the default compiler from Sun Studio to GCC 4.5.1.  This is described 
in more detail at [4].  In brief:

* You should change from your current compiler (Studio, GCC 3.4.3 or GCC 4.4) 
  to GCC 4.5.1, /opt/ts/bin/gcc.
* If you recompile any Studio- or GCC 3.4.3-compiled C++ code with the new 
  compiler, you need to recompile all of it, because the ABIs are not 
  compatible.
* GCC 3.4.3 will no longer be installed.
* If you use Studio-compiled versions of C++ libraries in 
  /opt/ts///, you should change to the GCC version in 
  /opt/ts//-gcc/.  

The following libraries require special handling:

* Studio-compiled versions of MySQL++ and VIPS were previously installed in 
  /opt/ts/lib.  If you use these libraries, you should switch to the version 
  that we will install in /opt/ts/ (where  is "mysqlpp" or "vips").  
  The Studio-compiled version will remain available for now.
* Studio-compiled versions of ImageMagick, sigc++ and cairomm are currently
  installed in /opt/ts.  They will be replaced with GCC-compiled versions
  in the same path, and Studio-compiled versions will not be available.
  If this adversely affects you (because you use these libraries and need time 
  to migrate), you should let us know before the maintenance.

[0] https://wiki.toolserver.org/view/Maintenance_schedule
[1] http://bugs.python.org/issue1054943
[2] http://bugs.python.org/issue10254
[3] http://lists.wikimedia.org/pipermail/toolserver-l/2010-November/003633.html
[4] 
http://lists.wikimedia.org/pipermail/toolserver-announce/2010-November/000367.html
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAkzz1HIACgkQIXd7fCuc5vIcAQCfffiUXe

Re: [Toolserver-l] Problem with matplolib / pylab

[Robin Krahl]

> This should be fixed now.

Thank you!

Regards,
Robin


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[Platonides]

Михајло Анђелковић wrote:
> Hello,
> 
> I would ask for allowance to run a request that can be resource
> consuming if not properly scaled:
> 
> SELECT page.page_title as title, rev_user_text as user, rev_timestamp
> as timestamp, rev_len as len FROM revision JOIN page ON page.page_id =
> rev_page WHERE rev_id > 0 AND rev_id < [...] AND rev_deleted = 0;
> 
> This is intended to extract basic data about all publicly visible
> revisions from 1 to [...]. Info about each revision would be a 4-tuple
> title/user name/time/length. I need this data to start generating a
> timeline of editing of srwiki, so it is intended to be run only once
> for each revision.
> 
> If this is generally allowed to do, my question is how large chunks of
> data can I take at once, and how long should be waited between two
> takes?
> 
> M

Have you considered generating the early timeline from dumps?

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Михајло Анђелковић:
> WHERE rev_id > 0 AND rev_id < [...] AND rev_deleted = 0;

Please check that MySQL plans this correctly (using the rev_id index).

> If this is generally allowed to do, my question is how large chunks of
> data can I take at once, and how long should be waited between two
> takes?

A simple fetch like this doesn't usually take many resources.  You can probably 
pick any random number for chunking.  If you run each chunk under SGE[0] as a 
separate job, and request an sqlprocs resource, SGE will handle scheduling for 
you so there's no need to worry about how long to wait between runs.

- river.

[0] https://wiki.toolserver.org/view/Job_scheduling
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (FreeBSD)

iEYEARECAAYFAkzzsQgACgkQIXd7fCuc5vI7ZACbBkmQAFXTSBEDYm+I8xf+Ci4X
LFQAnjOPp1zFUQ/Njs90nQ4us4IZswZE
=mPqu
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[Михајло Анђелковић]

Unfortunately, the complete dumps contain lots if data I don't
actually need and I am afraid I am not willing to commit such an
impact to my small HDD. And even more, they are really unavailable
since 10.11, which is kind of very long already.

Right now I have time for this research and I want to use toolserver,
unless admins do not let me do it. I asked per 'ask first'.

M

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Extracting basic revision data

[Petr Kadlec]

2010/11/29 Михајло Анђелковић :
> This is intended to extract basic data about all publicly visible
> revisions from 1 to [...]. Info about each revision would be a 4-tuple
> title/user name/time/length. I need this data to start generating a
> timeline of editing of srwiki, so it is intended to be run only once
> for each revision.

It seems to me you might want to use the dumps instead
, even though they are unavailable at
the moment…

-- [[ cs:User:Mormegil | Petr Kadlec ]]

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette