Re: [Toolserver-l] Needs of important ressources

[Seb35]

Ok, thank you, I didn't find this page.

For the BnF project we needed in fact about one day of computation (most  
of the time was used by the disk accesses), but we thought it would be  
more (we optimized too by using SAX instead of DOM to read big XML files,  
it used too much memory with DOM too).
For the video encoding to OGV (it's not me who done that), it was 4-5  
hours for a single video but some time was used to swap (and there are 100  
videos corresponding to the conferences).

Thank you for the response.
Seb35

Fri, 04 Mar 2011 03:45:53 +0100, MZMcBride  wrote:
> Seb35 wrote:
>> I'm from the French chapter and we need sometimes a lot of CPU power
>> and/or a lot of memory for some projects. For now it happened two times:
>> * the partnership with the BnF where we need CPU and disk space (for  
>> image
>> and DjVu processing, we rent a server dedicated to that);
>> * the treatment of the videos of our GLAM meeting (split the videos and
>> OGV conversion, done on a personal computer with a lot of time).
>>
>> So is it possible to use the toolserver in these cases? And/or ask for a
>> particular use when important ressources are required? I didn't seen
>> anything related to the ressources in the rules.
>
> It's difficult to know what "a lot" of CPU power or memory is from your
> post. Toolserver accounts have account limits
> (), so if you're staying
> within those limits, there's generally no problem. If you want to exceed
> those limits, you should talk to the Toolserver roots first
> (). There are  
> places
> like /mnt/user-store that can be used for large media storage as well.
>
> As always, the Toolserver resources that you use need to relate to  
> Wikimedia
> in some way, but it sounds like both of your projects do. :-)
>
> MZMcBride

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Needs of important ressources

[MZMcBride]

Seb35 wrote:
> I'm from the French chapter and we need sometimes a lot of CPU power
> and/or a lot of memory for some projects. For now it happened two times:
> * the partnership with the BnF where we need CPU and disk space (for image
> and DjVu processing, we rent a server dedicated to that);
> * the treatment of the videos of our GLAM meeting (split the videos and
> OGV conversion, done on a personal computer with a lot of time).
> 
> So is it possible to use the toolserver in these cases? And/or ask for a
> particular use when important ressources are required? I didn't seen
> anything related to the ressources in the rules.

It's difficult to know what "a lot" of CPU power or memory is from your
post. Toolserver accounts have account limits
(), so if you're staying
within those limits, there's generally no problem. If you want to exceed
those limits, you should talk to the Toolserver roots first
(). There are places
like /mnt/user-store that can be used for large media storage as well.

As always, the Toolserver resources that you use need to relate to Wikimedia
in some way, but it sounds like both of your projects do. :-)

MZMcBride



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Needs of important ressources

[Seb35]

Hello,

I'm from the French chapter and we need sometimes a lot of CPU power  
and/or a lot of memory for some projects. For now it happened two times:
* the partnership with the BnF where we need CPU and disk space (for image  
and DjVu processing, we rent a server dedicated to that);
* the treatment of the videos of our GLAM meeting (split the videos and  
OGV conversion, done on a personal computer with a lot of time).

So is it possible to use the toolserver in these cases? And/or ask for a  
particular use when important ressources are required? I didn't seen  
anything related to the ressources in the rules.

Thanks,
~ Seb35 [^_^]

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] user-store

[River Tarnell]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I'm currently copying user-store to a new disk array; this should be 
finished in a couple of days.  I hope the new array will be faster, as 
well as much larger.  There will be a separate maintenance notice for 
the actual switch later.

Once the move is done, in additional to having more storage now (about 
14TB), it will be much easier to add additional storage in the future.  
However, I don't think having a single 14+ TB filesystem is a good idea 
(the initial volume will be 5TB), so I'm considering new ways to split 
up the storage.

The most obvious is to have one volume for miscellaneous files, like the 
current user-store, and then one volume per large project: stats, dumps, 
etc.  People (or MMTs) who wanted additional storage for their project 
could request a new volume of a particular size.  As well as making it 
easier for us to account for space, this would mean you were guaranteed 
to get the amount of storage you asked for, unlike the current situation 
where the filesystem can (and does) fill up.

Does this seem like something that would be useful to users?  Otherwise, 
does anyone have another suggestions for how to allocate the space?

- river.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (NetBSD)

iEYEARECAAYFAk1wAlwACgkQIXd7fCuc5vKrKACfXttPL1HS1EzLxykm0t5/dggn
BzkAnj3KiTWGs8QR55sb0/Jv5XQY0uie
=nN4u
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette