Re: [Wikitech-l] w...@home Extension

2009-08-04 Thread Brion Vibber
On 8/3/09 9:56 PM, Gregory Maxwell wrote: [snip] Based on 'what other people do' I'd say the low should be in the 200kbit-300kbit/sec range. Perhaps taking the high up to a megabit? There are also a lot of very short videos on Wikipedia where the whole thing could reasonably be buffered

Re: [Wikitech-l] w...@home Extension

2009-08-03 Thread Michael Dale
perhaps if people create a lot of voice overs ~Kens burns~ effects on commons images with the occasional inter-spliced video clip with lots of back and forth editing... and we are constantly creating timely derivatives of these flattened sequences that ~may~ necessitate such a system..

Re: [Wikitech-l] w...@home Extension

2009-08-03 Thread Gregory Maxwell
On Mon, Aug 3, 2009 at 10:56 PM, Michael Dalemd...@wikimedia.org wrote: Also will hack in adding derivatives to the job queue where oggHandler is embed in a wiki-article at a substantial lower resolution than the source version. Will have it send the high res version until the derivative is

Re: [Wikitech-l] w...@home Extension

2009-08-02 Thread Marco Schuster
On Sun, Aug 2, 2009 at 2:32 AM, Platonides platoni...@gmail.com wrote: I'd actually be interested how YouTube and the other video hosters protect themselves against hacker threats - did they code totally new de/en-coders? That would be even more risky than using existing, tested

Re: [Wikitech-l] w...@home Extension

2009-08-02 Thread David Gerard
2009/8/2 Marco Schuster ma...@harddisk.is-a-geek.org: Really? If they simply don't publish the source (and the binaries), then the only possible way for an attacker is fuzzing... and that can take long time. I believe they use ffmpeg, like everyone does. The ffmpeg code has had people kicking

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Gregory Maxwell
On Sat, Aug 1, 2009 at 12:13 AM, Michael Dalemd...@wikimedia.org wrote: true... people will never upload to site without instant gratification ( cough youtube cough ) ... Hm? I just tried uploading to youtube and there was a video up right away. Other sizes followed within a minute or two. At

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Brian
On Sat, Aug 1, 2009 at 12:47 AM, Gregory Maxwell gmaxw...@gmail.com wrote: On Sat, Aug 1, 2009 at 12:13 AM, Michael Dalemd...@wikimedia.org wrote: Once you factor in the ratio of video to non-video content for the for-seeable future this comes off looking like a time wasting boondoggle. I

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Gregory Maxwell
On Sat, Aug 1, 2009 at 2:54 AM, Brianbrian.min...@colorado.edu wrote: On Sat, Aug 1, 2009 at 12:47 AM, Gregory Maxwell gmaxw...@gmail.com wrote: On Sat, Aug 1, 2009 at 12:13 AM, Michael Dalemd...@wikimedia.org wrote: Once you factor in the ratio of video to non-video content for the

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread David Gerard
2009/8/1 Brian brian.min...@colorado.edu: I think you vastly underestimate the amount of video that will be uploaded. Michael is right in thinking big and thinking distributed. CPU cycles are not *that* cheap. There is a lot of free video out there and as soon as we have a stable system in

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Kat Walsh
On Sat, Aug 1, 2009 at 9:57 AM, David Gerarddger...@gmail.com wrote: 2009/8/1 Brian brian.min...@colorado.edu: I think you vastly underestimate the amount of video that will be uploaded. Michael is right in thinking big and thinking distributed. CPU cycles are not *that* cheap. There is a lot

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Brian
On Sat, Aug 1, 2009 at 10:12 AM, Kat Walsh k...@mindspillage.org wrote: On Sat, Aug 1, 2009 at 9:57 AM, David Gerarddger...@gmail.com wrote: 2009/8/1 Brian brian.min...@colorado.edu: I think you vastly underestimate the amount of video that will be uploaded. Michael is right in thinking

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Brian
On Sat, Aug 1, 2009 at 10:17 AM, Brian brian.min...@colorado.edu wrote: On Sat, Aug 1, 2009 at 10:12 AM, Kat Walsh k...@mindspillage.org wrote: On Sat, Aug 1, 2009 at 9:57 AM, David Gerarddger...@gmail.com wrote: 2009/8/1 Brian brian.min...@colorado.edu: I think you vastly

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Gregory Maxwell
On Sat, Aug 1, 2009 at 12:17 PM, Brianbrian.min...@colorado.edu wrote: A reasonable estimate would require knowledge of how much free video can be automatically acquired, it's metadata automatically parsed and then automatically uploaded to commons. I am aware of some massive archives of free

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Brian
On Sat, Aug 1, 2009 at 11:04 AM, Gregory Maxwell gmaxw...@gmail.com wrote: On Sat, Aug 1, 2009 at 12:17 PM, Brianbrian.min...@colorado.edu wrote: A reasonable estimate would require knowledge of how much free video can be automatically acquired, it's metadata automatically parsed and then

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Gregory Maxwell
On Sat, Aug 1, 2009 at 1:13 PM, Brianbrian.min...@colorado.edu wrote: There are always tradeoffs. If I understand w...@home correctly it is also intended to be run @foundation. It works just as well for distributing transcoding over the foundation cluster as it does for distributing it to

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Michael Dale
Some notes: * ~its mostly an api~. We can run it internally if that is more cost efficient. ( will do on a command line client shortly ) ... (as mentioned earlier the present code was hacked together quickly its just a prototype. I will generalize things to work better as internal jobs. and I

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread David Gerard
2009/8/1 Brian brian.min...@colorado.edu: And of course, you can just ship them the binaries! Trusted clients are impossible. Particularly for prrotecting against lulz-seekers. - d. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Brian
On Sat, Aug 1, 2009 at 1:07 PM, David Gerard dger...@gmail.com wrote: 2009/8/1 Brian brian.min...@colorado.edu: And of course, you can just ship them the binaries! Trusted clients are impossible. Particularly for prrotecting against lulz-seekers. - d. Impossible? That's hyperbole.

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread David Gerard
2009/8/1 Brian brian.min...@colorado.edu: On Sat, Aug 1, 2009 at 1:07 PM, David Gerard dger...@gmail.com wrote: 2009/8/1 Brian brian.min...@colorado.edu: And of course, you can just ship them the binaries! Trusted clients are impossible. Particularly for prrotecting against lulz-seekers.

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Brian
On Sat, Aug 1, 2009 at 1:32 PM, David Gerard dger...@gmail.com wrote: 2009/8/1 Brian brian.min...@colorado.edu: On Sat, Aug 1, 2009 at 1:07 PM, David Gerard dger...@gmail.com wrote: 2009/8/1 Brian brian.min...@colorado.edu: And of course, you can just ship them the binaries! Trusted

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Marco Schuster
On Sat, Aug 1, 2009 at 9:35 PM, Brian brian.min...@colorado.edu wrote: Never trust the client. Ever, ever, ever. If you have a working model that relies on a trusted client you're fucked already. Basically, if you want to distribute binaries to reduce hackability ... it won't work and

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Mike.lifeguard
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 BTW, Who's idea was this extension? I know Michael Dale is writing it, but was this something assigned to him by someone else? Was it discussed beforehand? Or is this just Michael's project through and through? Thanks, - -Mike -BEGIN PGP

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Platonides
Marco Schuster wrote: What about video files exploiting some new 0day exploit in a video input format? The Wikimedia transcoding servers *must* be totally separated from the other WM servers to prevent 0wnage or a site-wide hack. That's not different than a 0day on tiff or djvu. You can do

Re: [Wikitech-l] w...@home Extension

2009-08-01 Thread Michael Dale
I had to program it anyway to support the distributing of the flattening of sequences. Which has been the planed approach for quite some time. I thought of the name and adding one-off support for transocoding recently, and hacked it up over the past few days. This code will eventually support

[Wikitech-l] w...@home Extension

2009-07-31 Thread Michael Dale
Want to point out the working prototype of the w...@home extension. Presently it focuses on a system for transcoding uploaded media to free formats, but will also be used for flattening sequences and maybe other things in the future ;) Its still rough around the edges ... it presently

Re: [Wikitech-l] w...@home Extension

2009-07-31 Thread Michael Dale
Gregory Maxwell wrote: On Fri, Jul 31, 2009 at 9:51 PM, Michael Dalemd...@wikimedia.org wrote: the transcode job into $wgChunkDuration length encoding jobs. ( each pieces is uploaded then reassembled on the server. that way big transcoding jobs can be distributed to as many clients that