On Dec 3, 2007 9:46 PM, Nathan Ingersoll <[EMAIL PROTECTED]> wrote:
> On 12/3/07, Gustavo Sverzut Barbieri <[EMAIL PROTECTED]> wrote:
> > On Dec 3, 2007 1:41 PM, Nathan Ingersoll <[EMAIL PROTECTED]> wrote:
> >
> > I'd like to have the minimum overhead with the safest approach, for
> > that I'd like to have just one "worker" process, if it crashes (or
> > take too long to reply) it would be restarted... but fork+exec
> > everytime is a no go... general usecase (not just canola) is to
> > thumbnail jpeg/png, these can be considered pretty safe/stable and
> > wouldn't cause much trouble... however using emotion to provide movie
> > thumbnails or an svg/edje reader may cause problems due the
> > flexibility to do bad things :-P
>
> Just to be clear, I'm not advocating fork+exec for each thumbnail for
> every case. My thinking is more along the lines that we keep thumbnail
> generation for core formats (jpeg, png) inside the primary worker
> thread. The difference is rather than requiring a new plugin for each
> format we want to support, allow for fork+exec style generation for
> specified file formats in order to re-use existing infrastructure.
> This could also be improved if some external handlers could take a
> batch of arguments. We can certainly provide plugins for formats that
> we support, which could improve efficiency, but we can't expect to
> support all file types.
ah, ok, this makes sense. I was afraid of one trying to call
imagimagick's "convert" to just process jpeg, but it make sense to use
it as a fallback (that app understand almost every format on earth).
> Thumbnailing is also a task that fits well in a parallel model, since
> the task has a fair amount of latency reading from disk. By using
> multiple workers you can take advantage of this latency and have CPU
> bound processes while others are IO bound. Also, since the tasks are
> independent they can be scaled out nicely across multiple cores. Both
> of these aspects may be irrelevant for Canola, since it is running off
> a flash based drive on single core CPU.
yes, I agree with parallel and most implementations (epsilon/e_thumb)
seems to support multiple workers, I'd like to keep it. As for it
doesn't make sense for us, _today_ it doesn't but there are multi-core
ARM comming, so it's good to think a bit ahead here :-)
> > I'm not really found of DBus, actually I was planning to have a
> > hand-written system to do the communication instead of ecore_{ipc,con}
> > because the messages are really simple and can be done to fit a fixed
> > size, much easier and fast to handle, also processing on peers could
> > avoid alloc/free and thus not trash the memory. If these are well
> > encapsulated like epsilon is today, I see no problem. Also, by having
> > such hand-written message system I'd not require ecore at all for the
> > server process, just the worker, it's good to have things as light as
> > possible.
>
> While I agree that keeping things light is important, I would like to
> know your reasoning behind why dbus is not a reasonable candidate in
> this case? I don't have any data to back it up, but I find it hard to
> believe that the communication overhead of dbus would be anywhere near
> the relative slowness of the actual thumbnail generation. As long as
> you can queue requests, the latency should be completely dominated by
> waiting for the resulting thumbnail. The IPC speed should really only
> be a problem if you are making round-trip requests and waiting on the
> result before the next request.
it can be, I also don't have any numbers to back me up... maybe I'm
just too paranoid after some problems we had with the device, things
that looked ok suddently turned into bootleneck, but with more time to
test I can prove myself wrong... these things always happen when I'm
under pressure :-P
> > I doubt it worth the pain, because if we go FDO we must remove any
> > attempt to use .edj as thumbnail format, and it's a good alternative
> > if you want to provide movie thumbnail. Going "too generic" may hurt
> > more than help if nobody else use it, and believe me these guys are
> > hard to get using done by others, you know the NIH syndrome...
>
> Except what you're encouraging is exactly NIH (not that we have any
> lack of that around here). What is the advantage of using edje for
> storing generated thumbnails vs using the filesystem? The main
> advantage of edje is efficient bundling of data into a single file,
> but with thumbnails, they are more tied to their original source than
> with other surrounding files. Also, edje writes out the entire eet
> each time a save is done, this means that you will now be writing out
> every enclosed thumbnail instead of the single one that is updated.
I was not clear here: I was saying to use edje to store each file, but
it was more target at movie thumbnails (see what rage_thumbd does).
But sure, we could use some mng/mjpeg/mpeg1 or another simple format.
> It's difficult to say if no one else would use it if we're not talking
> to anyone else about it. I think it is worth opening a discussion with
> FDO members with a base spec as a point of reference.
yes
> > As for INdT today, yes, it won't help, but it's good to discuss the
> > long term so we can help. Probably I'll hack some nasty epsilon2 to
> > get it done in time, but I'll not even commit to CVS, it'll stay in
> > staff.get-e.org GIT... after we have something good for the long term,
> > then I can help with it and then convert canola to use it.
>
> Oops, I got the capitalization scheme of INdT wrong again. This sounds
> like a reasonable plan for your short-term issue.
>
> Good luck with your release.
:-)
--
Gustavo Sverzut Barbieri
--------------------------------------
Jabber: [EMAIL PROTECTED]
MSN: [EMAIL PROTECTED]
ICQ#: 17249123
Skype: gsbarbieri
Mobile: +55 (81) 9927 0010
-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
enlightenment-devel mailing list
enlightenment-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
