[pygame] Camera module change proposal
Hello all, With 1.9 coming out soon, I think it is best that we decide on a final API for the camera module to use going forward. The current module was designed largely for v4l2 on low performance fixed point CPUs, and the API reflects that. With OS X support coming this summer from Werner, and Windows support coming soon, we should decide on something that works well across all platforms. 1. Remove colorspace conversion from the camera module: Currently, the camera module can capture images as RGB, YUV, or HSV. There are a lot of conversion shortcuts going on under the hood to get the quickest path from raw pixel data to the desired colorspace. Unfortunately, this also multiplies the amount of code and work necessary to bring the camera module to other platforms. The use cases for this feature are also fairly limited, since anyone doing serious computer vision is better off using OpenCV. I propose having the camera module do the more relevant task of just capturing RGB images, and have a seperate colorspace conversion function, either the transform or the color module, which takes any RGB surface and outputs an HSV or YUV surface. 2. Camera controls: I'm not sure what to do here. V4l2 controls are very useful to have, but there doesn't seem to be a Windows or OS X equivalent as far as I can tell. Any ideas on either? Nirav
Re: [pygame] Camera module change proposal
Hello Nirav, For #1, I can say that YUV colorspace is much faster than RGB to upload to the graphic card when using OpenGL. If for now it is only RGB that is implemented, it would surely be nice to have the possibilty to use YUV directly in a near future. For #2, on Mac OS X, there is at least the Quicktime Dialog that can offer a lot of fine controls on the image source and quality. It surely should be implemented in the first place. Regards, a 2009/5/18 Nirav Patel o...@spongezone.net: Hello all, With 1.9 coming out soon, I think it is best that we decide on a final API for the camera module to use going forward. The current module was designed largely for v4l2 on low performance fixed point CPUs, and the API reflects that. With OS X support coming this summer from Werner, and Windows support coming soon, we should decide on something that works well across all platforms. 1. Remove colorspace conversion from the camera module: Currently, the camera module can capture images as RGB, YUV, or HSV. There are a lot of conversion shortcuts going on under the hood to get the quickest path from raw pixel data to the desired colorspace. Unfortunately, this also multiplies the amount of code and work necessary to bring the camera module to other platforms. The use cases for this feature are also fairly limited, since anyone doing serious computer vision is better off using OpenCV. I propose having the camera module do the more relevant task of just capturing RGB images, and have a seperate colorspace conversion function, either the transform or the color module, which takes any RGB surface and outputs an HSV or YUV surface. 2. Camera controls: I'm not sure what to do here. V4l2 controls are very useful to have, but there doesn't seem to be a Windows or OS X equivalent as far as I can tell. Any ideas on either? Nirav -- Alexandre Quessy http://alexandre.quessy.net/
Re: [pygame] Camera module change proposal
On Tue, May 19, 2009 at 5:26 AM, Nirav Patel o...@spongezone.net wrote: Hello all, With 1.9 coming out soon, I think it is best that we decide on a final API for the camera module to use going forward. The current module was designed largely for v4l2 on low performance fixed point CPUs, and the API reflects that. With OS X support coming this summer from Werner, and Windows support coming soon, we should decide on something that works well across all platforms. We can't decide on a final API at this point. Better to wait until we find out what each one can do. Marking it final before getting into the implementation won't work. It's definitely time to discuss the API, just not marking it final until we need to. The camera module is going to be marked experimental, and we can change it post 1.9 - like we agreed. It's way to late in the 1.9 release process to change anything drastically in 1.9 now. 1. Remove colorspace conversion from the camera module: Currently, the camera module can capture images as RGB, YUV, or HSV. There are a lot of conversion shortcuts going on under the hood to get the quickest path from raw pixel data to the desired colorspace. Unfortunately, this also multiplies the amount of code and work necessary to bring the camera module to other platforms. The use cases for this feature are also fairly limited, since anyone doing serious computer vision is better off using OpenCV. I propose having the camera module do the more relevant task of just capturing RGB images, and have a seperate colorspace conversion function, either the transform or the color module, which takes any RGB surface and outputs an HSV or YUV surface. The idea of making as small amount of work as possible is a good one. Native formats + RGB + YUV from the camera object would be grand. some points: - people to get out the data as quickly as possible who need to (like if on low power machines, olpc, mobile phones etc). - zero extra coding for implementer if they are lazy, all the need to implement at minimum is RGB. - if the implementer wants to spend extra time making sure YUV is a fast path, they can. - yuv is important for Movie, and Overlay (two other important use cases). Having a colorspace conversion function in another module is fine (not for 1.9 though). However requiring it to be separate means the implementer can not output YUV etc in the fast path. If it is in the camera module, then if the implementor does not want to they can get out RGB, then convert it to YUV. They also have the option to try and get YUV out using a fast path. eg. this psedo code Camera class shows, how by keeping the details in the Camera class it's possible to get yuv in the fast path if implemented, and if not use the conversion function from RGB. class Camera(object): def get_yuv(self) if self.can_get_yuv: return self._yuv() else: rgb = self.get_rgb() return pygame.color.convert_rbg_to_yuv(rgb) The native formats should come simply as a buffer, and also information about that buffer. Details can be standardised as needed for these other types. However, the various RGB/yuv ones should be standardised. 2. Camera controls: I'm not sure what to do here. V4l2 controls are very useful to have, but there doesn't seem to be a Windows or OS X equivalent as far as I can tell. The general idea of camera controls(settings, properties, attributes etc) is available on every camera API. It should probably be generalised - rather than just the specific ones used in v4l2. camera = Camera() camera.flipx = 1 camera.fps = 10 camera.width = 320 camera.height = 200 Not sure exactly which controls will work Xplatorm at this stage... or if on every platform settings can be changed after the camera is started, or only before starting the camera. This is something that needs more research. Any ideas on either? Nirav
Re: [pygame] Camera module change proposal
On Mon, May 18, 2009 at 5:34 PM, René Dudfield ren...@gmail.com wrote: We can't decide on a final API at this point. Better to wait until we find out what each one can do. Marking it final before getting into the implementation won't work. It's definitely time to discuss the API, just not marking it final until we need to. The camera module is going to be marked experimental, and we can change it post 1.9 - like we agreed. It's way to late in the 1.9 release process to change anything drastically in 1.9 now. Alright, we can hold off until later. - yuv is important for Movie, and Overlay (two other important use cases). The YUV being output from the camera module is packed YUV, not the YUYV, YUV422, etc used for video or the Overlay module. It is really only useful for computer vision purposes. The get_raw() function in the camera module can however output data useful for Movie or Overlay. Having a colorspace conversion function in another module is fine (not for 1.9 though). However requiring it to be separate means the implementer can not output YUV etc in the fast path. If it is in the camera module, then if the implementor does not want to they can get out RGB, then convert it to YUV. They also have the option to try and get YUV out using a fast path. eg. this psedo code Camera class shows, how by keeping the details in the Camera class it's possible to get yuv in the fast path if implemented, and if not use the conversion function from RGB. class Camera(object): def get_yuv(self) if self.can_get_yuv: return self._yuv() else: rgb = self.get_rgb() return pygame.color.convert_rbg_to_yuv(rgb) This is basically how we are doing it now. The shorter path we are using now is initializing the camera with the pixelformat closest to the desired output. Eg: If you want YUV, it'll try to initialize with YUYV, or RGB565 for RGB. Once it's initialized, you would have to close and reopen it to switch between YUV and RGB to get a short path. Nirav
