[Toolserver-l] user-store & host sql (adenia) broken

2014-08-02 Thread Marlen Caemmerer
Hello, today adenia finally died - OS not found. Fortunatelly all relevant data was already on the SAN so I could move the DBs completely to cassia. The host alias sql also points there. user-store had to move as well and is now active on cassia, too. It was an interruption of about 2,5h all to

Re: [Toolserver-l] user-store and tile store failed again

2014-02-13 Thread Marlen Caemmerer
Hello, looks like the move of the file system improved performance and we are back to the old state. Cheers Marlen/nosy On Thu, 13 Feb 2014, Marlen Caemmerer wrote: Date: Thu, 13 Feb 2014 15:48:07 From: Marlen Caemmerer To: toolserver-l@lists.wikimedia.org Cc: map...@lists.wikimedi

[Toolserver-l] user-store and tile store failed again

2014-02-13 Thread Marlen Caemmerer
Hello, today the user-store on TS and tile store failed again. It is currently checking the file system and can take a while. I will move the stores to another host after this is done. Sorry for this, hope it gets better afterwards. Cheers Marlen/nosy __

Re: [Toolserver-l] user-store not mounted

2013-07-15 Thread Johannes Kroll
On Mon, 15 Jul 2013 11:17:04 +0200 (CEST) Marlen Caemmerer wrote: > Hey, > > sorry - fixed. When I moved user-store back to rosemary I forgot to update > the puppet entries Thanks. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.or

Re: [Toolserver-l] user-store not mounted

2013-07-15 Thread Marlen Caemmerer
Hey, sorry - fixed. When I moved user-store back to rosemary I forgot to update the puppet entries On Mon, 15 Jul 2013, Johannes Kroll wrote: Date: Mon, 15 Jul 2013 10:57:22 From: Johannes Kroll Reply-To: Wikimedia Toolserver To: Wikimedia Toolserver Subject: [Toolserver-l] user-store

[Toolserver-l] user-store not mounted

2013-07-15 Thread Johannes Kroll
Hi, /mnt/user-store is not mounted on ortelius. - J. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Maili

[Toolserver-l] user-store (temporary) back, willow will rebooted friday

2012-10-04 Thread DaB.
Hello all, like announced we have set up a temporary workaround for the user-store until hemlock is fixed. It should work now, the only exception is willow. I can't mount or umount the user-store on willow and the directory looks strange. So I hereby announce a reboot of willow for tomorrow, F

[Toolserver-l] user-store and TS-1230

2011-11-23 Thread John
Just a thought that I came up today. We currently use bz and tz compression for data in the stats directory. which is a about 2.3TB. If we switched to 7z compression we would gain at least a 25-30% gain in compression ratios. This seems one way to help address the lack of space. Another would be to

Re: [Toolserver-l] user-store

2011-03-07 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In article <4d74c74b.4010...@mathgen.ch>, Frederic Schutz wrote: > > * Dumps will go in /store/dumps// > > * Page view stats will go in /store/stats/ > > * Everything else will go in /store/misc/ > This looks good to me. I am ready to populate the st

Re: [Toolserver-l] user-store

2011-03-07 Thread Frederic Schutz
On 06/03/11 05:17, River Tarnell wrote: >> Once the move is done, in additional to having more storage now (about >> 14TB), it will be much easier to add additional storage in the future. >> However, I don't think having a single 14+ TB filesystem is a good idea >> (the initial volume will be 5TB)

Re: [Toolserver-l] user-store

2011-03-07 Thread Darkdadaah
> River Tarnell: > Well, no one commented on this, so I propose the following: > > * Dumps will go in /store/dumps// > * Page view stats will go in /store/stats/ > * Everything else will go in /store/misc/ Thanks a lot river. Yes, I think dividing in such a way is wise. Also, the current user-stor

Re: [Toolserver-l] user-store

2011-03-05 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 River Tarnell: > I'm currently copying user-store to a new disk array; this should be > finished in a couple of days. I hope the new array will be faster, as > well as much larger. There will be a separate maintenance notice for > the actual switc

[Toolserver-l] user-store maintenance, tonight

2011-03-04 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Later tonight (around 12AM UTC[0]) I will restart the NFS server on hemlock, which serves user-store. This will cause an interruption to service that should last for under 1 minute. - river. [0] http://time.tcx.org.uk/utc/2011-03-05/0

[Toolserver-l] user-store

2011-03-03 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, I'm currently copying user-store to a new disk array; this should be finished in a couple of days. I hope the new array will be faster, as well as much larger. There will be a separate maintenance notice for the actual switch later. Once the

Re: [Toolserver-l] user-store disk usage

2010-09-10 Thread Peter Körner
Am 10.09.2010 11:21, schrieb Kay Drangmeister: > Am 09.09.2010, 23:28 Uhr, schrieb Peter Körner: >> Am 09.09.2010 23:04, schrieb Colin Marquardt: >>> I always wanted to make the rendering dynamic (and then >>> never expire the tiles) >> It should not be to hard to write your own renderer. > > I wro

Re: [Toolserver-l] user-store disk usage

2010-09-10 Thread Kay Drangmeister
Am 09.09.2010, 23:28 Uhr, schrieb Peter Körner : > Am 09.09.2010 23:04, schrieb Colin Marquardt: >> I always wanted to make the rendering dynamic (and then >> never expire the tiles) > It should not be to hard to write your own renderer. I wrote a WMS server for this, it's easy. But I have the pr

Re: [Toolserver-l] user-store disk usage

2010-09-09 Thread Colin Marquardt
2010/9/9 Peter Körner : > > Am 09.09.2010 23:04, schrieb Colin Marquardt: >> I always wanted to make the rendering dynamic (and then >> never expire the tiles), but it would need a post-processing task in >> either renderd or tirex (the latter seems a bit more suited for that >> from what I read) f

Re: [Toolserver-l] user-store disk usage

2010-09-09 Thread Peter Körner
Am 09.09.2010 23:04, schrieb Colin Marquardt: > AIUI, River was just asking who *could* free space, not pushing > anybody to do so for useful stuff. I just wanted to mention that there's a need for them. > I always wanted to make the rendering dynamic (and then > never expire the tiles), but it

Re: [Toolserver-l] user-store disk usage

2010-09-09 Thread Colin Marquardt
2010/9/9 Peter Körner : > Am 09.09.2010 22:34, schrieb Colin Marquardt: >> >> These are the hillshading tiles used in e.g. http://hikebikemap.de > > It would be very sad if we would need to delete them. AIUI, River was just asking who *could* free space, not pushing anybody to do so for useful stu

Re: [Toolserver-l] user-store disk usage

2010-09-09 Thread Peter Körner
Am 09.09.2010 22:34, schrieb Colin Marquardt: > These are the hillshading tiles used in e.g. http://hikebikemap.de It would be very sad if we would need to delete them. Colin, how completed is the rendering of the hillshading tiles? I remember that it was still running on cassini when it got re-p

Re: [Toolserver-l] user-store disk usage

2010-09-09 Thread Colin Marquardt
2010/9/8 River Tarnell : > Here is the per-user usage report: > > cmarqu           : 492.62G in 126629282 files These are the hillshading tiles used in e.g. http://hikebikemap.de (and some other sites as well now). I had made a rather unsuccessful attempt in re-packing them into metatiles in the p

Re: [Toolserver-l] user-store disk usage

2010-09-08 Thread Daniel Schwen
> hemlock# du -sh iip-cache-dschwen > 201G    iip-cache-dschwen Oh, right, I forgot about that (checked it yesterday and thought in relation to the 3.7TB it was negligible...). That directory contains multiresolution images generated from very large images on commons for the zoomviewer [1]. Genera

Re: [Toolserver-l] user-store disk usage

2010-09-08 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Daniel Schwen: > Uhm, I see 215G in /mnt/user-store/wikiminiatlas/, I thought that is > the lions share of my stuff. Where is the other half? hemlock# du -sh iip-cache-dschwen 201Giip-cache-dschwen - river. -BEGIN PGP SIGNATURE-

Re: [Toolserver-l] user-store disk usage

2010-09-08 Thread Daniel Schwen
> dschwen          : 433.88G in 16212733 files Uhm, I see 215G in /mnt/user-store/wikiminiatlas/, I thought that is the lions share of my stuff. Where is the other half? I'm running my tile cache prune script now. It fell out of my crontab when I last moved my project to another server. _

Re: [Toolserver-l] user-store disk usage

2010-09-08 Thread Fabio Alessandro Locati
I've freed my 5.some gbs, since I don't use it any more On Wed, Sep 8, 2010 at 12:58 PM, River Tarnell wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Here is the per-user usage report: > > schutz           : 822.69G in 14604 files > cmarqu           : 492.62G in 126629282 files > mu

Re: [Toolserver-l] user-store disk usage

2010-09-08 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Here is the per-user usage report: schutz : 822.69G in 14604 files cmarqu : 492.62G in 126629282 files

Re: [Toolserver-l] user-store disk usage

2010-09-07 Thread Alex
On 9/7/2010 10:11 AM, Frederic Schutz wrote: > River Tarnell wrote: > >> If you are currently using disk space on user-store, please check your usage >> and delete any unneeded files. There is only 70GB of space left on the >> volume >> now. >> >> (I will produce a more detailed report of who'

Re: [Toolserver-l] user-store disk usage

2010-09-07 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Frederic Schutz: > ==> over 20% of the data is mine... This is only 20%, and as far as I know does not grow at the rate that would explain the recent increase in user-store disk usage. Still, if you find no one is using the older files, and they ar

Re: [Toolserver-l] user-store disk usage

2010-09-07 Thread Frederic Schutz
River Tarnell wrote: > If you are currently using disk space on user-store, please check your usage > and delete any unneeded files. There is only 70GB of space left on the > volume > now. > > (I will produce a more detailed report of who's using space later...) I can save you some time:

[Toolserver-l] user-store disk usage

2010-09-07 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, If you are currently using disk space on user-store, please check your usage and delete any unneeded files. There is only 70GB of space left on the volume now. (I will produce a more detailed report of who's using space later...) - ri

Re: [Toolserver-l] user-store maintenance

2009-09-08 Thread Aryeh Gregor
On Tue, Sep 8, 2009 at 3:52 AM, Andrew Dunbar wrote: > OK. There seems to be some permission problems though. I can't create a new > file > at /mnt/user-store/ This should be fixed now. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)

Re: [Toolserver-l] user-store maintenance

2009-09-08 Thread Andrew Dunbar
2009/9/8 River Tarnell : > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Andrew Dunbar: >> Carefuly you've accidentally mounted it as /mnt/user-store/user-store/ > > actually the mount was correct, but the contents of the directory were > wrong.. this should be fixed now. OK. There seems to

Re: [Toolserver-l] user-store maintenance

2009-09-08 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrew Dunbar: > Carefuly you've accidentally mounted it as /mnt/user-store/user-store/ actually the mount was correct, but the contents of the directory were wrong.. this should be fixed now. - river. -BEGIN PGP SIGNATURE- Version: G

Re: [Toolserver-l] user-store maintenance

2009-09-08 Thread Andrew Dunbar
2009/9/8 River Tarnell : > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > due to bad timing this took a little bit longer than expected, but it is > now finished and user-store is available again. Carefuly you've accidentally mounted it as /mnt/user-store/user-store/ Cheers. Andrew Dunbar (h

Re: [Toolserver-l] user-store maintenance

2009-09-08 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 due to bad timing this took a little bit longer than expected, but it is now finished and user-store is available again. - river. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (HP-UX) iEYEARECAAYFAkqmAmgACgkQIXd7fCuc5vJ8FwCgiRqGM7Jedeut

[Toolserver-l] user-store maintenance

2009-09-07 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 hi, i've unmounted /mnt/user-store in order to convert hemlock from ZFS to VxFS. this involves copying off the data, reformatting the disk, then copying the data back; i estimate it will take around 12 hours in total, during which user-store will be