[Toolserver-l] user-store & host sql (adenia) broken

[Marlen Caemmerer]

Hello,

today adenia finally died - OS not found.
Fortunatelly all relevant data was already on the SAN so I could move the DBs 
completely to cassia. The host alias sql also points there.
user-store had to move as well and is now active on cassia, too.

It was an interruption of about 2,5h all together.

This regards very few users only but you should know since OSM and Merls tools were not 
accessible and are probably still broken if they dont use the hostname "sql" 
for the queries.


Cheers
nosy


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store and tile store failed again

[Marlen Caemmerer]

Hello,

looks like the move of the file system improved performance and we are back to 
the old state.

Cheers
Marlen/nosy


On Thu, 13 Feb 2014, Marlen Caemmerer wrote:


Date: Thu, 13 Feb 2014 15:48:07
From: Marlen Caemmerer 
To: toolserver-l@lists.wikimedia.org
Cc: map...@lists.wikimedia.org
Subject: user-store and tile store failed again

Hello,

today the user-store on TS and tile store failed again.
It is currently checking the file system and can take a while.
I will move the stores to another host after this is done.
Sorry for this, hope it gets better afterwards.

Cheers
Marlen/nosy





___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store and tile store failed again

[Marlen Caemmerer]

Hello,

today the user-store on TS and tile store failed again.
It is currently checking the file system and can take a while.
I will move the stores to another host after this is done.
Sorry for this, hope it gets better afterwards.

Cheers
Marlen/nosy


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store not mounted

[Johannes Kroll]

On Mon, 15 Jul 2013 11:17:04 +0200 (CEST)
Marlen Caemmerer  wrote:

> Hey,
> 
> sorry - fixed. When I moved user-store back to rosemary I forgot to update 
> the puppet entries

Thanks.



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store not mounted

[Marlen Caemmerer]

Hey,

sorry - fixed. When I moved user-store back to rosemary I forgot to update the 
puppet entries

On Mon, 15 Jul 2013, Johannes Kroll wrote:


Date: Mon, 15 Jul 2013 10:57:22
From: Johannes Kroll 
Reply-To: Wikimedia Toolserver 
To: Wikimedia Toolserver 
Subject: [Toolserver-l] user-store not mounted

Hi,

/mnt/user-store is not mounted on ortelius.

- J.



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store not mounted

[Johannes Kroll]

Hi,

/mnt/user-store is not mounted on ortelius.

 - J.



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store (temporary) back, willow will rebooted friday

[DaB.]

Hello all,

like announced we have set up a temporary workaround for the user-store until 
hemlock is fixed. It should work now, the only exception is willow. I can't 
mount or umount the user-store on willow and the directory looks strange. So I 
hereby announce a reboot of willow for tomorrow,

Friday, 16:00 UTC.

The downtime should be 10 minutes. You can follow the process at [1].

[1] https://jira.toolserver.org/browse/MNT-1272

-- 
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885


signature.asc
Description: This is a digitally signed message part.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store and TS-1230

[John]

Just a thought that I came up today. We currently use bz and tz compression
for data in the stats directory. which is a about 2.3TB. If we switched to
7z compression we would gain at least a 25-30% gain in compression ratios.
This seems one way to help address the lack of space. Another would be to
make it public what directories are eating up the most space.

John
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

In article <4d74c74b.4010...@mathgen.ch>,
Frederic Schutz   wrote:
> > * Dumps will go in /store/dumps//
> > * Page view stats will go in /store/stats/
> > * Everything else will go in /store/misc/
> This looks good to me. I am ready to populate the stats system as soon 
> as it is ready to be used. Have you already decided the size ? It could 
> do with 3 Tb...

Can you estimate the rate of growth for the stats?  That would help in 
planning the size.

The change probably won't happen immediately, since our current (free) 
VxVM license only allows 4 volumes.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk11JxoACgkQIXd7fCuc5vI6sQCeODn6bahLCSI0uyzL19Kgyw+j
E4UAoKryjpQeBZ70jgYmmjFjgZA/083g
=lT6X
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store

[Frederic Schutz]

On 06/03/11 05:17, River Tarnell wrote:

>> Once the move is done, in additional to having more storage now (about
>> 14TB), it will be much easier to add additional storage in the future.
>> However, I don't think having a single 14+ TB filesystem is a good idea
>> (the initial volume will be 5TB), so I'm considering new ways to split
>> up the storage.
>
> Well, no one commented on this, so I propose the following:

Sorry, I was off the Internet for the past few days.

> * Dumps will go in /store/dumps//
> * Page view stats will go in /store/stats/
> * Everything else will go in /store/misc/
>
> Symlinks from the old locations will prevent breaking tools which access
> data in its current location.  Projects which need a lot of disk space
> (say, 100GB+) will be able to request a volume at /store// for
> their files.

This looks good to me. I am ready to populate the stats system as soon 
as it is ready to be used. Have you already decided the size ? It could 
do with 3 Tb...

Frédéric

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store

[Darkdadaah]

> River Tarnell:
> Well, no one commented on this, so I propose the following:
>
> * Dumps will go in /store/dumps//
> * Page view stats will go in /store/stats/
> * Everything else will go in /store/misc/
Thanks a lot river. Yes, I think dividing in such a way is wise.

Also, the current user-store is kinda messy, with lots of dumps here
and there, and some files we don't really know what they are for.
Maybe it is the right time to clean up a bit.

If I may, I'd strongly urge anyone that have files there to create
explicit directories for their project and indicate what these files
are for in the INDEX (as indicated in README).
I would also like all the dump files to be properly, hierarchically
stored like /store/dumps//dump_files.

That would make things easier (and cleaner).

Thanks again,
Darkdadaah

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

River Tarnell:
> I'm currently copying user-store to a new disk array; this should be 
> finished in a couple of days.  I hope the new array will be faster, as 
> well as much larger.  There will be a separate maintenance notice for 
> the actual switch later.
 
This is done now; in the end I did the switch without a maintenance.  
Some data is still copying (should be done in about 4 hours), so it'll 
be slower until then.

> Once the move is done, in additional to having more storage now (about 
> 14TB), it will be much easier to add additional storage in the future.  
> However, I don't think having a single 14+ TB filesystem is a good idea 
> (the initial volume will be 5TB), so I'm considering new ways to split 
> up the storage.

Well, no one commented on this, so I propose the following:

* Dumps will go in /store/dumps//
* Page view stats will go in /store/stats/
* Everything else will go in /store/misc/

Symlinks from the old locations will prevent breaking tools which access 
data in its current location.  Projects which need a lot of disk space 
(say, 100GB+) will be able to request a volume at /store// for 
their files.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1zCtwACgkQIXd7fCuc5vJwIACgmNFo5NYc0PENM0V7UQp8npLo
o/EAoJP0JWAZm7leh7xdFB0QA2Qqn2eQ
=HIOq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store maintenance, tonight

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

Later tonight (around 12AM UTC[0]) I will restart the NFS server on 
hemlock, which serves user-store.  This will cause an interruption to 
service that should last for under 1 minute.

- river.

[0] http://time.tcx.org.uk/utc/2011-03-05/00:00
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1xNqIACgkQIXd7fCuc5vL22QCeM2IXEqx0x3yRrcbXIbjVbCsW
GR0An1hYPDoUq58rs1G9fSxgTrT4P1h1
=4BKf
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I'm currently copying user-store to a new disk array; this should be 
finished in a couple of days.  I hope the new array will be faster, as 
well as much larger.  There will be a separate maintenance notice for 
the actual switch later.

Once the move is done, in additional to having more storage now (about 
14TB), it will be much easier to add additional storage in the future.  
However, I don't think having a single 14+ TB filesystem is a good idea 
(the initial volume will be 5TB), so I'm considering new ways to split 
up the storage.

The most obvious is to have one volume for miscellaneous files, like the 
current user-store, and then one volume per large project: stats, dumps, 
etc.  People (or MMTs) who wanted additional storage for their project 
could request a new volume of a particular size.  As well as making it 
easier for us to account for space, this would mean you were guaranteed 
to get the amount of storage you asked for, unlike the current situation 
where the filesystem can (and does) fill up.

Does this seem like something that would be useful to users?  Otherwise, 
does anyone have another suggestions for how to allocate the space?

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1wAlwACgkQIXd7fCuc5vKrKACfXttPL1HS1EzLxykm0t5/dggn
BzkAnj3KiTWGs8QR55sb0/Jv5XQY0uie
=nN4u
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Peter Körner]

Am 10.09.2010 11:21, schrieb Kay Drangmeister:
> Am 09.09.2010, 23:28 Uhr, schrieb Peter Körner:
>> Am 09.09.2010 23:04, schrieb Colin Marquardt:
>>> I always wanted to make the rendering dynamic (and then
>>> never expire the tiles)
>> It should not be to hard to write your own renderer.
>
> I wrote a WMS server for this, it's easy.
>
> But I have the problems that with mod_tile+tirex, my tiles always
> expire, and I don't know why. I've set all timeouts to dozens of weeks,
> but they keep expiring after 1 day or so.
As far as I understood the process, the expiry is done by mod_tile [1]. 
You might need to patch that out somehow.

It it really is mod_tile, then you should see messages in your apache 
log when you set the apache log-level [2] to debug.

Peter


[1] 

[2] 

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Kay Drangmeister]

Am 09.09.2010, 23:28 Uhr, schrieb Peter Körner :
> Am 09.09.2010 23:04, schrieb Colin Marquardt:
>> I always wanted to make the rendering dynamic (and then
>> never expire the tiles)
> It should not be to hard to write your own renderer.

I wrote a WMS server for this, it's easy.

But I have the problems that with mod_tile+tirex, my tiles always
expire, and I don't know why. I've set all timeouts to dozens of weeks,
but they keep expiring after 1 day or so.

It MIGHT have something to do that I purge and fill my database
daily, and so it assumes that the data changed, and triggers a
dirty state, but I found neigher indication of this, nor documentation,
nor log entries, debugging possibilities or similar.

So I consider switching back to tilecache, sadly.

Kind regards,
Kay

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Colin Marquardt]

2010/9/9 Peter Körner :
>
> Am 09.09.2010 23:04, schrieb Colin Marquardt:
>> I always wanted to make the rendering dynamic (and then
>> never expire the tiles), but it would need a post-processing task in
>> either renderd or tirex (the latter seems a bit more suited for that
>> from what I read) for the alpha-channel stuff, and ideally do the
>> metatiling in the same step.
>
> It should not be to hard to write your own renderer. You may want to take a
> look at
>
> 
> and
> 

Thanks, that sounds doable. As winter draws nearer, time at home will
increase :)

Cheers
  Colin

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Peter Körner]

Am 09.09.2010 23:04, schrieb Colin Marquardt:
> AIUI, River was just asking who *could* free space, not pushing
> anybody to do so for useful stuff.
I just wanted to mention that there's a need for them.

> I always wanted to make the rendering dynamic (and then
> never expire the tiles), but it would need a post-processing task in
> either renderd or tirex (the latter seems a bit more suited for that
> from what I read) for the alpha-channel stuff, and ideally do the
> metatiling in the same step.
It should not be to hard to write your own renderer. You may want to 
take a look at


and


Peter

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Colin Marquardt]

2010/9/9 Peter Körner :
> Am 09.09.2010 22:34, schrieb Colin Marquardt:
>>
>> These are the hillshading tiles used in e.g. http://hikebikemap.de
>
> It would be very sad if we would need to delete them.

AIUI, River was just asking who *could* free space, not pushing
anybody to do so for useful stuff.

> Colin, how completed is the rendering of the hillshading tiles? I remember
> that it was still running on cassini when it got re-purposed.

I don't remember exactly, I got Europe and a part of Asia down to zoom
15, with parts down to 16 I think, and North America to zoom 14 or
some such. I always wanted to make the rendering dynamic (and then
never expire the tiles), but it would need a post-processing task in
either renderd or tirex (the latter seems a bit more suited for that
from what I read) for the alpha-channel stuff, and ideally do the
metatiling in the same step.

Cheers
  Colin

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Peter Körner]

Am 09.09.2010 22:34, schrieb Colin Marquardt:
> These are the hillshading tiles used in e.g. http://hikebikemap.de
It would be very sad if we would need to delete them.

Colin, how completed is the rendering of the hillshading tiles? I 
remember that it was still running on cassini when it got re-purposed.

Peter

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Colin Marquardt]

2010/9/8 River Tarnell :
> Here is the per-user usage report:
>
> cmarqu           : 492.62G in 126629282 files

These are the hillshading tiles used in e.g. http://hikebikemap.de
(and some other sites as well now). I had made a rather unsuccessful
attempt in re-packing them into metatiles in the past, which I could
try another time if the number of files became a problem (which you
said wasn't, back then).

Cheers
  Colin

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Daniel Schwen]

> hemlock# du -sh iip-cache-dschwen
> 201G    iip-cache-dschwen

Oh, right, I forgot about that (checked it yesterday and thought in
relation to the 3.7TB it was negligible...).
That directory contains multiresolution images generated from very
large images on commons for the zoomviewer [1].
Generating them takes up to more than a minute each, and many of them
actually had to be generated on an external server (as the process
uses a lot of memory for a short time and is frequently killed by
slayerd). Deleting them is not really an option (well it is, but it
would mean discontinuing the service).
Daniel

[1] http://toolserver.org/~dschwen/iip/wip.php?f=Seattle_7.jpg

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Daniel Schwen:
> Uhm, I see 215G in /mnt/user-store/wikiminiatlas/, I thought that is
> the lions share of my stuff. Where is the other half?

hemlock# du -sh iip-cache-dschwen 
201Giip-cache-dschwen

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.16 (FreeBSD)

iEYEARECAAYFAkyHnVAACgkQIXd7fCuc5vKWNQCfb+GeooSEjFExInjX2nyORXsE
LXQAnjYihAJvKbdud/P+Fnw6xMdOwKE6
=3O7y
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Daniel Schwen]

> dschwen          : 433.88G in 16212733 files

Uhm, I see 215G in /mnt/user-store/wikiminiatlas/, I thought that is
the lions share of my stuff. Where is the other half?
I'm running my tile cache prune script now. It fell out of my crontab
when I last moved my project to another server.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Fabio Alessandro Locati]

I've freed my 5.some gbs, since I don't use it any more

On Wed, Sep 8, 2010 at 12:58 PM, River Tarnell
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Here is the per-user usage report:
>
> schutz           : 822.69G in 14604 files
> cmarqu           : 492.62G in 126629282 files
> multichill       : 475.83G in 2261720 files
> dschwen          : 433.88G in 16212733 files
> sk               : 169.82G in    83 files
> emijrp           :  82.75G in   583 files
> cbm              :  49.34G in 10035 files
> bryan            :  19.57G in 1162796 files
> drh08            :  17.33G in  1837 files
> dispenser        :  15.03G in     2 files
> johang           :  12.09G in  2404 files
> mzmcbride        :  11.66G in     2 files
> hippietrail      :   7.55G in    30 files
> seb35            :   7.00G in    28 files
> enwp10           :   6.09G in    17 files
> fale             :   5.32G in     1 files
> contests         :   3.11G in  1293 files
> holek            :   2.96G in     1 files
> pathoschild      :   2.61G in     1 files
> saper            :   2.51G in 112507 files
> liangent         :   2.44G in     1 files
> autocracy        :   2.12G in     1 files
> valhallasw       :   1.81G in     1 files
> enwikt           :   1.49G in     5 files
> danny_b          :   1.34G in    19 files
> daniel           : 489.88M in     1 files
> magnus           : 430.12M in  1150 files
> vyznev           : 269.83M in     1 files
> balu             : 206.10M in     2 files
> mauro742         :  38.48M in     1 files
> overlordq        :   3.19M in     1 files
>
>        - river.
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v2.0.16 (FreeBSD)
>
> iEYEARECAAYFAkyHbEsACgkQIXd7fCuc5vLisACeO1hmNfXxIF9fSywBWuzhHyLI
> 5dgAn3C4v9a9vItsIfzG8oMa5Sq5H2ok
> =Jrtq
> -END PGP SIGNATURE-
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>



-- 
Fabio Alessandro Locati

Home: Segrate, Milan, Italy (GMT +1)
Phone: +39-328-3799681
MSN/Jabber/E-Mail: fabioloc...@gmail.com

PGP Fingerprint: 5525 8555 213C 19EB 25F2  A047 2AD2 BE67 0F01 CA61

Involved in: KDE, OpenStreetMap, Ubuntu, Wikimedia

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Here is the per-user usage report:

schutz   : 822.69G in 14604 files   
 
cmarqu   : 492.62G in 126629282 files   
 
multichill   : 475.83G in 2261720 files 
 
dschwen  : 433.88G in 16212733 files
 
sk   : 169.82G in83 files   
 
emijrp   :  82.75G in   583 files   
 
cbm  :  49.34G in 10035 files   
 
bryan:  19.57G in 1162796 files 
 
drh08:  17.33G in  1837 files   
 
dispenser:  15.03G in 2 files   
 
johang   :  12.09G in  2404 files   
 
mzmcbride:  11.66G in 2 files   
 
hippietrail  :   7.55G in30 files   
 
seb35:   7.00G in28 files   
 
enwp10   :   6.09G in17 files   
 
fale :   5.32G in 1 files   
 
contests :   3.11G in  1293 files   
 
holek:   2.96G in 1 files   
 
pathoschild  :   2.61G in 1 files   
 
saper:   2.51G in 112507 files  
 
liangent :   2.44G in 1 files   
 
autocracy:   2.12G in 1 files   
 
valhallasw   :   1.81G in 1 files   
 
enwikt   :   1.49G in 5 files   
 
danny_b  :   1.34G in19 files   
 
daniel   : 489.88M in 1 files   
 
magnus   : 430.12M in  1150 files   
 
vyznev   : 269.83M in 1 files   
 
balu : 206.10M in 2 files   
 
mauro742 :  38.48M in 1 files   
 
overlordq:   3.19M in 1 files   
 

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.16 (FreeBSD)

iEYEARECAAYFAkyHbEsACgkQIXd7fCuc5vLisACeO1hmNfXxIF9fSywBWuzhHyLI
5dgAn3C4v9a9vItsIfzG8oMa5Sq5H2ok
=Jrtq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Alex]

On 9/7/2010 10:11 AM, Frederic Schutz wrote:
> River Tarnell wrote:
> 
>> If you are currently using disk space on user-store, please check your usage 
>> and delete any unneeded files.  There is only 70GB of space left on the 
>> volume 
>> now.
>>
>> (I will produce a more detailed report of who's using space later...)
> 
> I can save you some time:
> 
>$ du -h /mnt/user-store/stats
>822G   /mnt/user-store/stats
> 
> ==> over 20% of the data is mine...
> 
> This contains the raw data for the consultation statistics for all 
> projects. It can easily be cleaned, since there is data starting in 
> January 2009 in there. But I'd be glad to know who uses it, and if 
> people only use the most recent files, or if some of them use the 
> archives -- so that I know what can be cleaned (it could even be done 
> automatically).

I use these, mainly only the newest ones. Though I'll need all of
January 2010 rather soon.

> I remember there were discussions about having some dedicated space for 
> these files (and a more accessible version of them), I don't know if 
> this went anywhere ?
> 
> Just in case, I keep a copy of all older files, and can provide them to 
> anyone who would need them. In the recent past, I did not want to remove 
> them, because there were not too many copies of them around, and I was 
> glad to have a copy on the toolserver, but the WMF has now downloaded a 
> copy of them, so I don't mind so much deleting them

Are these publicly accessible somewhere? Domas's site only keeps the
last few months - http://dammit.lt/wikistats/archive/2010/

-- 
Alex (wikipedia:en:User:Mr.Z-man)

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Frederic Schutz:
> ==> over 20% of the data is mine...

This is only 20%, and as far as I know does not grow at the rate that would 
explain the recent increase in user-store disk usage.  Still, if you find no 
one is using the older files, and they are available somewhere else, they could 
be deleted.

> I remember there were discussions about having some dedicated space for 
> these files (and a more accessible version of them), I don't know if 
> this went anywhere ?

We are working on this at the moment.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.16 (FreeBSD)

iEYEARECAAYFAkyGSKYACgkQIXd7fCuc5vIMGQCdHaKYVlWj04py+AEoXutgtFit
frQAn2D8vFTOH6cgj8HsnemnP0I51zxJ
=7Kko
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Frederic Schutz]

River Tarnell wrote:

> If you are currently using disk space on user-store, please check your usage 
> and delete any unneeded files.  There is only 70GB of space left on the 
> volume 
> now.
> 
> (I will produce a more detailed report of who's using space later...)

I can save you some time:

   $ du -h /mnt/user-store/stats
   822G /mnt/user-store/stats

==> over 20% of the data is mine...

This contains the raw data for the consultation statistics for all 
projects. It can easily be cleaned, since there is data starting in 
January 2009 in there. But I'd be glad to know who uses it, and if 
people only use the most recent files, or if some of them use the 
archives -- so that I know what can be cleaned (it could even be done 
automatically).

I remember there were discussions about having some dedicated space for 
these files (and a more accessible version of them), I don't know if 
this went anywhere ?

Just in case, I keep a copy of all older files, and can provide them to 
anyone who would need them. In the recent past, I did not want to remove 
them, because there were not too many copies of them around, and I was 
glad to have a copy on the toolserver, but the WMF has now downloaded a 
copy of them, so I don't mind so much deleting them

Frédéric

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store disk usage

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

If you are currently using disk space on user-store, please check your usage 
and delete any unneeded files.  There is only 70GB of space left on the volume 
now.

(I will produce a more detailed report of who's using space later...)

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.16 (FreeBSD)

iEYEARECAAYFAkyGPtIACgkQIXd7fCuc5vIiPQCgv7cTh9bV2gPP9TFhr8K2ZlQG
vRYAoL05w9TKADrJ70KEvtNjCGVhctjr
=0JWw
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store maintenance

[Aryeh Gregor]

On Tue, Sep 8, 2009 at 3:52 AM, Andrew Dunbar  wrote:
> OK. There seems to be some permission problems though. I can't create a new 
> file
> at /mnt/user-store/

This should be fixed now.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store maintenance

[Andrew Dunbar]

2009/9/8 River Tarnell :
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Andrew Dunbar:
>> Carefuly you've accidentally mounted it as /mnt/user-store/user-store/
>
> actually the mount was correct, but the contents of the directory were
> wrong.. this should be fixed now.

OK. There seems to be some permission problems though. I can't create a new file
at /mnt/user-store/

Andrew Dunbar (hippietrail)

>        - river.
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (HP-UX)
>
> iEYEARECAAYFAkqmChkACgkQIXd7fCuc5vJ+0gCgifRlrTVIIJsS+DnhggpGtKeX
> lDAAnR0+D10aC3fnrJOyqwGxWYuvrBhY
> =SwGq
> -END PGP SIGNATURE-
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>



-- 
http://wiktionarydev.leuksman.com http://linguaphile.sf.net

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store maintenance

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Andrew Dunbar:
> Carefuly you've accidentally mounted it as /mnt/user-store/user-store/

actually the mount was correct, but the contents of the directory were
wrong.. this should be fixed now.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (HP-UX)

iEYEARECAAYFAkqmChkACgkQIXd7fCuc5vJ+0gCgifRlrTVIIJsS+DnhggpGtKeX
lDAAnR0+D10aC3fnrJOyqwGxWYuvrBhY
=SwGq
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store maintenance

[Andrew Dunbar]

2009/9/8 River Tarnell :
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> due to bad timing this took a little bit longer than expected, but it is
> now finished and user-store is available again.

Carefuly you've accidentally mounted it as /mnt/user-store/user-store/

Cheers. Andrew Dunbar (hippietrail)

>        - river.
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (HP-UX)
>
> iEYEARECAAYFAkqmAmgACgkQIXd7fCuc5vJ8FwCgiRqGM7Jedeut5Iq0T4fxcX79
> mEMAoJqnuahT+vG8j8mOy5sAXHzpVBFH
> =e8hA
> -END PGP SIGNATURE-
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>



-- 
http://wiktionarydev.leuksman.com http://linguaphile.sf.net

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store maintenance

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

due to bad timing this took a little bit longer than expected, but it is
now finished and user-store is available again.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (HP-UX)

iEYEARECAAYFAkqmAmgACgkQIXd7fCuc5vJ8FwCgiRqGM7Jedeut5Iq0T4fxcX79
mEMAoJqnuahT+vG8j8mOy5sAXHzpVBFH
=e8hA
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store maintenance

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

hi,

i've unmounted /mnt/user-store in order to convert hemlock from ZFS to
VxFS.  this involves copying off the data, reformatting the disk, then
copying the data back; i estimate it will take around 12 hours in total,
during which user-store will be unavailable.

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (HP-UX)

iEYEARECAAYFAkqk3lkACgkQIXd7fCuc5vJz+ACgodImzIzJrqIavmf8E/SoS1QW
BfEAn0yeyexjWw/UTGHVJvDLLZ3u8LkG
=eDpb
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette