Re: Idea of a v4l - fb interface driver
On Sat, May 29, 2010 at 6:06 AM, Ville Syrjälä syrj...@sci.fi wrote: On Fri, May 28, 2010 at 03:41:46PM -0400, Alex Deucher wrote: On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat If he wants different (independent) content on each output, just provide multiple /dev/fbX devices. I admit that we could use a controlling interface here that decides which user (application) might draw at a time to the interface which they currently only do if they are the active VT. If you want 2 or more outputs to be merged as one just configure this in the driver. The only thing that is impossible to do in fbdev is controlling 2 or more independent display outputs that access the same buffer. But that's not an issue I think. The things above only could use a unification of how to set them up on module load time (as only limited runtime changes are permited given that we must always be able to support a mode that we once entered during runtime). What about changing outputs on the fly (turn off VGA, turn on DVI, switch between multi-head and single-head, etc) or encoders shared between multiple connectors (think a single dac shared between a VGA and a TV port); how do you expose them easily as separate fbdevs? Lots of stuff is doable with fbdev, but it's nicer with kms. But actually getting your data onto the screen is a lot easier with fbdev. There's no standard API in drm to actually allocate the framebuffer and manipulate it. You always need a user space driver to go along with the kernel bits. I'm not saying fbdev is better than drm/kms but at least it can be used to write simple applications that work across different hardware. Perhaps that's something that should be addressed in the drm API. http://www.mail-archive.com/dri-de...@lists.sourceforge.net/msg48461.html I started writing a dumb API for KMS, it got stuck on whether to expose cursors or not, I must dig out the branch. It basically was a create + map API. I'll see if I can get it finished off. The main reason we avoided a fully generic interface is there isn't really a generic way to abstract acceleration on a modern GPU, and buffer allocation on most modern GPUs doesn't want a linear simple buffer. We felt doing a compromised generic interface would lead people down the wrong path into believing they could easily move from the dumb interface to a real accelerated one. There is a userspace library called libkms that abstracts this stuff, but I'd like to just have the kernel do it. Dave. -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Thu, 27 May 2010, Alex Deucher wrote: On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: ... Ok, let me explain what exactly I meant. Above I referred to display drivers, which is not the same as a framebuffer controller driver or whatever you would call it. By framebuffer controller driver I mean a driver for the actual graphics engine on a certain graphics card or an SoC. This is the part, that reads data from the actual framebuffer and outputs it to some hardware interface to a display device. Now this interface can be a VGA or a DVI connector, it can be one of several bus types, used with various LCD displays. In many cases this is all you have to do to get the output to your display. But in some cases the actual display on the other side of this bus also requires a driver. That can be some kind of a smart display, it can be a panel with an attached display controller, that must be at least configured, say, over SPI, it can be a display, attached to the host over the MIPI DSI bus, and implementing some proprietary commands. In each of these cases you will have to write a display driver for this specific display or controller type, and your framebuffer driver will have to interface with that display driver. Now, obviously, those displays can be connected to a variety of host systems, in which case you will want to reuse that display driver. This means, there has to be a standard fb-driver - display-driver API. AFAICS, this is currently not implemented in fbdev, please, correct me if I am wrong. Another API to consider in the drm kms (kernel modesetting) interface. The kms API deals properly with advanced display hardware and properly handles crtcs, encoders, and connectors. It also provides fbdev api emulation. Well, is KMS planned as a replacement for both fbdev and user-space graphics drivers? I mean, if you'd be writing a new fb driver for a relatively simple embedded SoC, would KMS apriori be a preferred API? Thanks Guennadi --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
(re-adding lists to CC) On Thu, 27 May 2010, Rob Clark wrote: Hi Guennadi, Sounds like an interesting idea... but how about the inverse? A v4l2 interface on top of fbdev. If v4l2 was more widely available as an output device, perhaps more userspace software would utilize it. Don't see any advantage in doing this apart from attracting user-space developers to develop for v4l2 output interface, which doesn't seem like a worthy goal in itself. Whereas with my translation you get access to existing user-space applications and to a powerful in-kernel API, and achieve a better interoperability with video-input streams. Thanks Guennadi --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: On Thu, 27 May 2010, Alex Deucher wrote: On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: ... Ok, let me explain what exactly I meant. Above I referred to display drivers, which is not the same as a framebuffer controller driver or whatever you would call it. By framebuffer controller driver I mean a driver for the actual graphics engine on a certain graphics card or an SoC. This is the part, that reads data from the actual framebuffer and outputs it to some hardware interface to a display device. Now this interface can be a VGA or a DVI connector, it can be one of several bus types, used with various LCD displays. In many cases this is all you have to do to get the output to your display. But in some cases the actual display on the other side of this bus also requires a driver. That can be some kind of a smart display, it can be a panel with an attached display controller, that must be at least configured, say, over SPI, it can be a display, attached to the host over the MIPI DSI bus, and implementing some proprietary commands. In each of these cases you will have to write a display driver for this specific display or controller type, and your framebuffer driver will have to interface with that display driver. Now, obviously, those displays can be connected to a variety of host systems, in which case you will want to reuse that display driver. This means, there has to be a standard fb-driver - display-driver API. AFAICS, this is currently not implemented in fbdev, please, correct me if I am wrong. Another API to consider in the drm kms (kernel modesetting) interface. The kms API deals properly with advanced display hardware and properly handles crtcs, encoders, and connectors. It also provides fbdev api emulation. Well, is KMS planned as a replacement for both fbdev and user-space graphics drivers? I mean, if you'd be writing a new fb driver for a relatively simple embedded SoC, would KMS apriori be a preferred API? It's become the defacto standard for X and things like EGL are being built onto of the API. As for the kms vs fbdev, kms provides a nice API for complex display setups with multiple display controllers and connectors while fbdev assumes one monitor/connector/encoder per device. The fbdev and console stuff has yet to take advantage of this flexibility, I'm not sure what will happen there. fbdev emulation is provided by kms, but it has to hide the complexity of the attached displays. For an soc with a single encoder and display, there's probably not much advantage over fbdev, but if you have an soc that can do TMDS and LVDS and possibly analog tv out, it gets more interesting. drm has historically been tied to pci, but Jordan Crouse recently posted changes to support platform devices: http://lists.freedesktop.org/archives/dri-devel/2010-May/000887.html Alex -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
Alex Deucher schrieb: On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: On Thu, 27 May 2010, Alex Deucher wrote: Another API to consider in the drm kms (kernel modesetting) interface. The kms API deals properly with advanced display hardware and properly handles crtcs, encoders, and connectors. It also provides fbdev api emulation. Well, is KMS planned as a replacement for both fbdev and user-space graphics drivers? I mean, if you'd be writing a new fb driver for a relatively simple embedded SoC, would KMS apriori be a preferred API? It's become the defacto standard for X and things like EGL are being built onto of the API. As for the kms vs fbdev, kms provides a nice API for complex display setups with multiple display controllers and connectors while fbdev assumes one monitor/connector/encoder per device. The fbdev and console stuff has yet to take advantage of this flexibility, I'm not sure what will happen there. fbdev emulation is provided by kms, but it has to hide the complexity of the attached displays. For an soc with a single encoder and display, there's probably not much advantage over fbdev, but if you have an soc that can do TMDS and LVDS and possibly analog tv out, it gets more interesting. Well hiding complexity is actually the job of an API. I don't see any need for major changes in fbdev for complex display setups. In most cases as a userspace application you really don't want to be bothered how many different output devices you have and control each individually, you just want an area to draw and to know/control what area the user is expected to see and that's already provided in fbdev. If the user wants the same content on multiple outputs just configure the driver to do so. If he wants different (independent) content on each output, just provide multiple /dev/fbX devices. I admit that we could use a controlling interface here that decides which user (application) might draw at a time to the interface which they currently only do if they are the active VT. If you want 2 or more outputs to be merged as one just configure this in the driver. The only thing that is impossible to do in fbdev is controlling 2 or more independent display outputs that access the same buffer. But that's not an issue I think. The things above only could use a unification of how to set them up on module load time (as only limited runtime changes are permited given that we must always be able to support a mode that we once entered during runtime). The thing that's really missing in fbdev is a way to allow hardware acceleration for userspace. Regards, Florian Tobias Schandinat -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Fri, 28 May 2010, Florian Tobias Schandinat wrote: Well hiding complexity is actually the job of an API. I don't see any need for major changes in fbdev for complex display setups. In most cases as a userspace application you really don't want to be bothered how many different output devices you have and control each individually, you just want an area to draw and to know/control what area the user is expected to see and that's already provided in fbdev. If the user wants the same content on multiple outputs just configure the driver to do so. If he wants different (independent) content on each output, just provide multiple /dev/fbX devices. I admit that we could use a controlling interface here that decides which user (application) might draw at a time to the interface which they currently only do if they are the active VT. If you want 2 or more outputs to be merged as one just configure this in the driver. The only thing that is impossible to do in fbdev is controlling 2 or more independent display outputs that access the same buffer. But that's not an issue I think. The things above only could use a unification of how to set them up on module load time (as only limited runtime changes are permited given that we must always be able to support a mode that we once entered during runtime). The thing that's really missing in fbdev is a way to allow hardware acceleration for userspace. How about a simple use-case, that I asked about in another my mail: how do you inform fbdev users, if a (DVI) display has been disconnected and another one with a different resolution has been connected? Thanks Guennadi --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat florianschandi...@gmx.de wrote: Alex Deucher schrieb: On Fri, May 28, 2010 at 4:21 AM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: On Thu, 27 May 2010, Alex Deucher wrote: Another API to consider in the drm kms (kernel modesetting) interface. The kms API deals properly with advanced display hardware and properly handles crtcs, encoders, and connectors. It also provides fbdev api emulation. Well, is KMS planned as a replacement for both fbdev and user-space graphics drivers? I mean, if you'd be writing a new fb driver for a relatively simple embedded SoC, would KMS apriori be a preferred API? It's become the defacto standard for X and things like EGL are being built onto of the API. As for the kms vs fbdev, kms provides a nice API for complex display setups with multiple display controllers and connectors while fbdev assumes one monitor/connector/encoder per device. The fbdev and console stuff has yet to take advantage of this flexibility, I'm not sure what will happen there. fbdev emulation is provided by kms, but it has to hide the complexity of the attached displays. For an soc with a single encoder and display, there's probably not much advantage over fbdev, but if you have an soc that can do TMDS and LVDS and possibly analog tv out, it gets more interesting. Well hiding complexity is actually the job of an API. I don't see any need for major changes in fbdev for complex display setups. In most cases as a userspace application you really don't want to be bothered how many different output devices you have and control each individually, you just want an area to draw and to know/control what area the user is expected to see and that's already provided in fbdev. Users want to be able to dynamically change their display config on the fly. If the user wants the same content on multiple outputs just configure the driver to do so. KMS provide an API to do that and a nice internal abstraction for handling it. If he wants different (independent) content on each output, just provide multiple /dev/fbX devices. I admit that we could use a controlling interface here that decides which user (application) might draw at a time to the interface which they currently only do if they are the active VT. If you want 2 or more outputs to be merged as one just configure this in the driver. The only thing that is impossible to do in fbdev is controlling 2 or more independent display outputs that access the same buffer. But that's not an issue I think. The things above only could use a unification of how to set them up on module load time (as only limited runtime changes are permited given that we must always be able to support a mode that we once entered during runtime). What about changing outputs on the fly (turn off VGA, turn on DVI, switch between multi-head and single-head, etc) or encoders shared between multiple connectors (think a single dac shared between a VGA and a TV port); how do you expose them easily as separate fbdevs? Lots of stuff is doable with fbdev, but it's nicer with kms. Alex The thing that's really missing in fbdev is a way to allow hardware acceleration for userspace. Regards, Florian Tobias Schandinat -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
Guennadi Liakhovetski schrieb: On Fri, 28 May 2010, Florian Tobias Schandinat wrote: Well hiding complexity is actually the job of an API. I don't see any need for major changes in fbdev for complex display setups. In most cases as a userspace application you really don't want to be bothered how many different output devices you have and control each individually, you just want an area to draw and to know/control what area the user is expected to see and that's already provided in fbdev. If the user wants the same content on multiple outputs just configure the driver to do so. If he wants different (independent) content on each output, just provide multiple /dev/fbX devices. I admit that we could use a controlling interface here that decides which user (application) might draw at a time to the interface which they currently only do if they are the active VT. If you want 2 or more outputs to be merged as one just configure this in the driver. The only thing that is impossible to do in fbdev is controlling 2 or more independent display outputs that access the same buffer. But that's not an issue I think. The things above only could use a unification of how to set them up on module load time (as only limited runtime changes are permited given that we must always be able to support a mode that we once entered during runtime). The thing that's really missing in fbdev is a way to allow hardware acceleration for userspace. How about a simple use-case, that I asked about in another my mail: how do you inform fbdev users, if a (DVI) display has been disconnected and another one with a different resolution has been connected? Yes that's a problem. The thing is that the virtual terminal requires us to always be able to switch to a resolution we once supported. Probably what I would do in such a case is switching the screen off and let the user figure out that he has done something wrong. As we don't really know our users (applications) there is not much we can do but wait for the next check_var to solve this. So yes things that force us to be incompatible with our previous behaviour can do some harm if the user is not aware of it. Note 1: Interesting that you mentioned viafb, the driver that is currently nearly completely incapable to determine any output device limitations. Note 2: set_par returns a value but that's rather a mistake and we can't rely anyone to react on a sane way on error. check_var is the only place where framebuffers can say that they don't support something after this they have to support it regardless of any external events. Thanks, Florian Tobias Schandinat -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Fri, May 28, 2010 at 03:41:46PM -0400, Alex Deucher wrote: On Fri, May 28, 2010 at 3:15 PM, Florian Tobias Schandinat If he wants different (independent) content on each output, just provide multiple /dev/fbX devices. I admit that we could use a controlling interface here that decides which user (application) might draw at a time to the interface which they currently only do if they are the active VT. If you want 2 or more outputs to be merged as one just configure this in the driver. The only thing that is impossible to do in fbdev is controlling 2 or more independent display outputs that access the same buffer. But that's not an issue I think. The things above only could use a unification of how to set them up on module load time (as only limited runtime changes are permited given that we must always be able to support a mode that we once entered during runtime). What about changing outputs on the fly (turn off VGA, turn on DVI, switch between multi-head and single-head, etc) or encoders shared between multiple connectors (think a single dac shared between a VGA and a TV port); how do you expose them easily as separate fbdevs? Lots of stuff is doable with fbdev, but it's nicer with kms. But actually getting your data onto the screen is a lot easier with fbdev. There's no standard API in drm to actually allocate the framebuffer and manipulate it. You always need a user space driver to go along with the kernel bits. I'm not saying fbdev is better than drm/kms but at least it can be used to write simple applications that work across different hardware. Perhaps that's something that should be addressed in the drm API. -- Ville Syrjälä syrj...@sci.fi http://www.sci.fi/~syrjala/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Idea of a v4l - fb interface driver
-Original Message- From: linux-fbdev-ow...@vger.kernel.org [mailto:linux-fbdev- ow...@vger.kernel.org] On Behalf Of Guennadi Liakhovetski Sent: Wednesday, May 26, 2010 7:40 PM To: linux-fb...@vger.kernel.org Subject: Idea of a v4l - fb interface driver This message has been earlier sent to the V4L mailing list Linux Media Mailing List linux-media@vger.kernel.org I replied to Linux-Media list, but saw this thread again. So pasting my response here and adding linux-media to CC. When replying, please add it to CC. This is a discussion proposal for the V4L mini summit, that is going to take place in June in Helsinki, Finland. Any opinions very welcome. V4L(2) video output vs. framebuffer. Problem: Currently the standard way to provide graphical output on various (embedded) displays like LCDs is to use a framebuffer driver. The interface is well supported and widely adopted in the user-space, many applications, including the X-server, various libraries like directfb, gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a number of problems. It is unmaintained. The infrastructure is not being further developed, every specific hardware driver is being supported by the respective architecture community. But as video output hardware evolves, more complex displays and buses appear and have to be supported, the subsystem shows its aging. For example, there is currently no way to write reusable across multiple platforms display drivers. [Hiremath, Vaibhav] Up to certain extent yes you are correct. OTOH V4L2 has a standard video output driver support, it is not very widely used, in the userspace I know only of gstreamer, that somehow supports video-output v4l2 devices in latest versions. But, being a part of the v4l2 subsystem, these drivers already now can take a full advantage of all v4l2 APIs, including the v4l2-subdev API for the driver reuse. So, how can we help graphics driver developers on the one hand by providing them with a capable driver framework (v4l2) and on the other hand by simplifying the task of interfacing to the user-space? [Hiremath, Vaibhav] I think this is really complex question which requires healthy discussion over list. How about a v4l2-output - fbdev translation layer? You write a v4l2-output driver and get a framebuffer device free of charge... TBH, I haven't given this too much of a thought, but so far I don't see anything that would make this impossible in principle. The video buffer management is quite different between the two systems, but maybe we can teach video-output drivers to work with just one buffer too? [Hiremath, Vaibhav] I believe V4L2 buf won't limit you to do this. Atleast in case of OMAP v4L2 display driver we are sticking to last buffer if application fails to queue one. So for me this is single buffer keeps on displaying unless application queue next buffer. Anyway, feel free to tell me why this is an absolutely impossible / impractical idea;) [Hiremath, Vaibhav] If I understanding correctly you are trying to propose something like, Without changing Fbdev interface to user space application, create translation layers which will allow driver developer to write driver under V4L2 framework providing /dev/fbx but using V4L2 API/framework. Also as mentioned by Jaya, it would be great if you put benefits we are targeting would be helpful. Thanks, Vaibhav Thanks Guennadi --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line unsubscribe linux-fbdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
(adding V4L ML to CC and preserving the complete reply for V4L readers) On Thu, 27 May 2010, Jaya Kumar wrote: On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: Problem: Currently the standard way to provide graphical output on various (embedded) displays like LCDs is to use a framebuffer driver. The interface is well supported and widely adopted in the user-space, many applications, including the X-server, various libraries like directfb, gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a number of problems. It is unmaintained. The infrastructure is not being further developed, every specific hardware driver is being supported by the respective architecture community. But as video output hardware I understand the issue you are raising, but to be clear there are several developers, Geert, Krzysztof, and others who are helping with the role of fbdev maintainer while Tony is away. If you meant that it has no specific currently active maintainer person, when you wrote unmaintained, then I agree that is correct. Exactly, I just interpreted this excerpt from MAINTAINERS: FRAMEBUFFER LAYER L: linux-fb...@vger.kernel.org W: http://linux-fbdev.sourceforge.net/ S: Orphan We're not sure where Tony is and we hope he's okay and that he'll be back soon. But if you meant that it is not maintained as in bugs aren't being fixed, then I'd have to slightly disagree. Maybe not as fast as commercial organizations seem to think should come for free, but still they are being worked on. evolves, more complex displays and buses appear and have to be supported, the subsystem shows its aging. For example, there is currently no way to write reusable across multiple platforms display drivers. At first I misread your point as talking about multi-headed displays which you're correct is not so great in fbdev. But write reusable across multi-platform display driver, I did not understand fully. I maintain a fbdev driver, broadsheetfb, that we're using on arm and x86 without problems and my presumption is other fbdev drivers are also capable of this unless the author made it explicitly platform specific. Ok, let me explain what exactly I meant. Above I referred to display drivers, which is not the same as a framebuffer controller driver or whatever you would call it. By framebuffer controller driver I mean a driver for the actual graphics engine on a certain graphics card or an SoC. This is the part, that reads data from the actual framebuffer and outputs it to some hardware interface to a display device. Now this interface can be a VGA or a DVI connector, it can be one of several bus types, used with various LCD displays. In many cases this is all you have to do to get the output to your display. But in some cases the actual display on the other side of this bus also requires a driver. That can be some kind of a smart display, it can be a panel with an attached display controller, that must be at least configured, say, over SPI, it can be a display, attached to the host over the MIPI DSI bus, and implementing some proprietary commands. In each of these cases you will have to write a display driver for this specific display or controller type, and your framebuffer driver will have to interface with that display driver. Now, obviously, those displays can be connected to a variety of host systems, in which case you will want to reuse that display driver. This means, there has to be a standard fb-driver - display-driver API. AFAICS, this is currently not implemented in fbdev, please, correct me if I am wrong. In my experience with adding defio to the fbdev infra, the fbdev community seemed quite good and I did not notice any aging problems. I realize there's probably issues that you're encountering where fbdev might be weak, this is good, and if you raise them specifically, I think the community can work together to address the issues. OTOH V4L2 has a standard video output driver support, it is not very widely used, in the userspace I know only of gstreamer, that somehow supports video-output v4l2 devices in latest versions. But, being a part of the v4l2 subsystem, these drivers already now can take a full advantage of all v4l2 APIs, including the v4l2-subdev API for the driver reuse. So, how can we help graphics driver developers on the one hand by providing them with a capable driver framework (v4l2) and on the other hand by simplifying the task of interfacing to the user-space? I think it would help if there were more specific elaborations on the functionality that you'd want the fbdev community to improve and how it could reuse v4l2 for this. Since some time the V4L2 kernel driver API includes a v4l2-subdev API, which is used to interface between drivers for various single components. Typical examples are USB camera bridges and camera sensors in webcams,
RE: Idea of a v4l - fb interface driver
On Thu, 27 May 2010, Hiremath, Vaibhav wrote: OTOH V4L2 has a standard video output driver support, it is not very widely used, in the userspace I know only of gstreamer, that somehow supports video-output v4l2 devices in latest versions. But, being a part of the v4l2 subsystem, these drivers already now can take a full advantage of all v4l2 APIs, including the v4l2-subdev API for the driver reuse. So, how can we help graphics driver developers on the one hand by providing them with a capable driver framework (v4l2) and on the other hand by simplifying the task of interfacing to the user-space? [Hiremath, Vaibhav] I think this is really complex question which requires healthy discussion over list. So, let's do that;) How about a v4l2-output - fbdev translation layer? You write a v4l2-output driver and get a framebuffer device free of charge... TBH, I haven't given this too much of a thought, but so far I don't see anything that would make this impossible in principle. The video buffer management is quite different between the two systems, but maybe we can teach video-output drivers to work with just one buffer too? [Hiremath, Vaibhav] I believe V4L2 buf won't limit you to do this. Atleast in case of OMAP v4L2 display driver we are sticking to last buffer if application fails to queue one. So for me this is single buffer keeps on displaying unless application queue next buffer. Good, so, that's not a problem. Anyway, feel free to tell me why this is an absolutely impossible / impractical idea;) [Hiremath, Vaibhav] If I understanding correctly you are trying to propose something like, Without changing Fbdev interface to user space application, create translation layers which will allow driver developer to write driver under V4L2 framework providing /dev/fbx but using V4L2 API/framework. Exactly. Also as mentioned by Jaya, it would be great if you put benefits we are targeting would be helpful. One of the benefits is the availability the subdevice API, and the forthcoming media controller API. I think, on some SoCs graphics processing units (scalers, format converters, compressors / decompressors) can be configured to either video input or output paths, so, it would make sense to manage them from one (v4l) driver framework. And I don't see a reason why you cannot have a /dev/fbX interface to the user-space at the same time. Yes, you can code it into your v4l driver, and some drivers do that already, but why not have it once for all? Last but not lease, having multiple incompatible subsystems in the kernel for pretty much the same task seems somewhat redundant to me. Yes, I know, this is not a complete redundancy, both v4l and fbdev have features, unsupported by the other, but IMHO some redundancy is definitely there. Thanks Guennadi --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Thu, May 27, 2010 at 2:56 PM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: (adding V4L ML to CC and preserving the complete reply for V4L readers) On Thu, 27 May 2010, Jaya Kumar wrote: On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: Ok, let me explain what exactly I meant. Above I referred to display drivers, which is not the same as a framebuffer controller driver or whatever you would call it. By framebuffer controller driver I mean a driver for the actual graphics engine on a certain graphics card or an SoC. This is the part, that reads data from the actual framebuffer and outputs it to some hardware interface to a display device. Now this interface can be a VGA or a DVI connector, it can be one of several bus types, used with various LCD displays. In many cases this is all you have to do to get the output to your display. But in some cases the actual display on the other side of this bus also requires a driver. That can be some kind of a smart display, it can be a panel with an attached display controller, that must be at least configured, say, over SPI, it can be a display, attached to the host over the MIPI DSI bus, and implementing some proprietary commands. In each of these cases you will have to write a display driver for this specific display or controller type, and your framebuffer driver will have to interface with that display driver. Now, obviously, those displays can be connected to a variety of host systems, in which case you will want to reuse that display driver. This means, there has to be a standard fb-driver - display-driver API. AFAICS, this is currently not implemented in fbdev, please, correct me if I am wrong. Thanks Guennadi, your clarification is useful. Yes, you are correct. There is no general fbdev API provided so that a host controller driver and a independent display panel driver can interface in a clean abstracted way. You've raised the MIPI-DSI issue. It is a good area to focus the discussion on for fbdev minded people and one that needs to be resolved soon so that we don't get dozens of host controller specific mipi display panel drivers. I had seen that omap2 fbdev has a portion of the MIPI-DSI command set exposed to their various display panel drivers which then hands off these commands to the omap specific lcd_mipid.c which uses spi. I see you've also implemented a similar concept in sh-mobile. When I saw the multiple display panel drivers showing up in omap, I raised a concern with Tomi and I think there was an intent to try to improve the abstraction. I'm not sure how far that has progressed. Are you saying v4l would help us in that area? I'm not yet able to follow the details of how using v4l would help address the need for mipi-dsi abstraction. Could you elaborate on that? Thanks, jaya -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Thu, 27 May 2010, Jaya Kumar wrote: You've raised the MIPI-DSI issue. It is a good area to focus the discussion on for fbdev minded people and one that needs to be resolved soon so that we don't get dozens of host controller specific mipi display panel drivers. I had seen that omap2 fbdev has a portion of the MIPI-DSI command set exposed to their various display panel drivers which then hands off these commands to the omap specific lcd_mipid.c which uses spi. I see you've also implemented a similar concept in sh-mobile. When I saw the multiple display panel drivers showing up in omap, I raised a concern with Tomi and I think there was an intent to try to improve the abstraction. I'm not sure how far that has progressed. Are you saying v4l would help us in that area? I'm not yet able to follow the details of how using v4l would help address the need for mipi-dsi abstraction. Could you elaborate on that? Well, I thought about an abstract driver for MIPI DSI... But, there is not really much there, that you can abstract. I've created a generic mipi_display.h header, that contains defines for display related (DSI, DCS) commands and transaction types. Once this header is in the mainline, we plan to convert OMAP drivers to it too. To talk to MIPI displays you need a capability to send and receive generic short and long telegrams, so, providing higher level functions like get_display_id() or soft_reset(), probably, wouldn't make sense. What you do need a proper API for is, when you start supporting proprietary display-specific commands and want to reuse those display drivers with different MIPI DSI hosts. For that we will want a generic API like .send_short_command(), .send_short_command_param(), etc. As for using v4l2 for MIPI displays - well, I am not sure it makes sense at all. This could make sense if, e.g., you were writing a driver for a graphics controller, capable to talk to various PHYs over a fixed bus (which is actually also the case with the sh-mobile LCDC), then you could design it, using V4L2, in the following way: /dev/videoX/dev/fbX | | | /- - fbdev translate - -/ v v v4l2 output device driver | v v4l2-subdev API |...| v v MIPI PHY ... parallel PHY driver ...driver | v MIPI bus abstraction | v MIPI display driver So, you would use the v4l2-subdev API to abstract various PHY drivers. The /dev/fbX link above would, certainly, only exist if we implement the v4l2-output - fbdev translation driver. Thanks Guennadi --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
Am 27.05.2010 08:44, schrieb Hiremath, Vaibhav: V4L(2) video output vs. framebuffer. Problem: Currently the standard way to provide graphical output on various (embedded) displays like LCDs is to use a framebuffer driver. The interface is well supported and widely adopted in the user-space, many applications, including the X-server, various libraries like directfb, gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a number of problems. It is unmaintained. The infrastructure is not being further developed, every specific hardware driver is being supported by the respective architecture community. But as video output hardware evolves, more complex displays and buses appear and have to be supported, the subsystem shows its aging. For example, there is currently no way to write reusable across multiple platforms display drivers. To add another point of view: I'm not completely sure how much these topics overlap, but another area where there's display output available, but not using some generic interface like fbdev, are DVB adapters with video output capabilities, e.g. /dev/dvb/adapterX/osdY devices and similar. The 'old' style Technotrend Full Featured DVB cards can only display either mpeg streams or very basic and restricted OSD overlays, but the newer generation of HD capable video decoders are usually capable of displaying an RGB32 video overlay in HD resolution. If these decoder cards would provide a framebuffer device, these devices could instantly be used for various media applications on the TV, like xbmc. Actually, the missing ability to run generic apps on DVB output devices is one of their biggest disadvantage over regular graphics cards with video acceleration. Maybe such a v4l-fbdev interface could handle such devices too? Cheers, Udo -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Idea of a v4l - fb interface driver
On Thu, May 27, 2010 at 2:56 AM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: (adding V4L ML to CC and preserving the complete reply for V4L readers) On Thu, 27 May 2010, Jaya Kumar wrote: On Wed, May 26, 2010 at 10:09 PM, Guennadi Liakhovetski g.liakhovet...@gmx.de wrote: Problem: Currently the standard way to provide graphical output on various (embedded) displays like LCDs is to use a framebuffer driver. The interface is well supported and widely adopted in the user-space, many applications, including the X-server, various libraries like directfb, gstreamer, mplayer, etc. In the kernel space, however, the subsystem has a number of problems. It is unmaintained. The infrastructure is not being further developed, every specific hardware driver is being supported by the respective architecture community. But as video output hardware I understand the issue you are raising, but to be clear there are several developers, Geert, Krzysztof, and others who are helping with the role of fbdev maintainer while Tony is away. If you meant that it has no specific currently active maintainer person, when you wrote unmaintained, then I agree that is correct. Exactly, I just interpreted this excerpt from MAINTAINERS: FRAMEBUFFER LAYER L: linux-fb...@vger.kernel.org W: http://linux-fbdev.sourceforge.net/ S: Orphan We're not sure where Tony is and we hope he's okay and that he'll be back soon. But if you meant that it is not maintained as in bugs aren't being fixed, then I'd have to slightly disagree. Maybe not as fast as commercial organizations seem to think should come for free, but still they are being worked on. evolves, more complex displays and buses appear and have to be supported, the subsystem shows its aging. For example, there is currently no way to write reusable across multiple platforms display drivers. At first I misread your point as talking about multi-headed displays which you're correct is not so great in fbdev. But write reusable across multi-platform display driver, I did not understand fully. I maintain a fbdev driver, broadsheetfb, that we're using on arm and x86 without problems and my presumption is other fbdev drivers are also capable of this unless the author made it explicitly platform specific. Ok, let me explain what exactly I meant. Above I referred to display drivers, which is not the same as a framebuffer controller driver or whatever you would call it. By framebuffer controller driver I mean a driver for the actual graphics engine on a certain graphics card or an SoC. This is the part, that reads data from the actual framebuffer and outputs it to some hardware interface to a display device. Now this interface can be a VGA or a DVI connector, it can be one of several bus types, used with various LCD displays. In many cases this is all you have to do to get the output to your display. But in some cases the actual display on the other side of this bus also requires a driver. That can be some kind of a smart display, it can be a panel with an attached display controller, that must be at least configured, say, over SPI, it can be a display, attached to the host over the MIPI DSI bus, and implementing some proprietary commands. In each of these cases you will have to write a display driver for this specific display or controller type, and your framebuffer driver will have to interface with that display driver. Now, obviously, those displays can be connected to a variety of host systems, in which case you will want to reuse that display driver. This means, there has to be a standard fb-driver - display-driver API. AFAICS, this is currently not implemented in fbdev, please, correct me if I am wrong. Another API to consider in the drm kms (kernel modesetting) interface. The kms API deals properly with advanced display hardware and properly handles crtcs, encoders, and connectors. It also provides fbdev api emulation. Alex In my experience with adding defio to the fbdev infra, the fbdev community seemed quite good and I did not notice any aging problems. I realize there's probably issues that you're encountering where fbdev might be weak, this is good, and if you raise them specifically, I think the community can work together to address the issues. OTOH V4L2 has a standard video output driver support, it is not very widely used, in the userspace I know only of gstreamer, that somehow supports video-output v4l2 devices in latest versions. But, being a part of the v4l2 subsystem, these drivers already now can take a full advantage of all v4l2 APIs, including the v4l2-subdev API for the driver reuse. So, how can we help graphics driver developers on the one hand by providing them with a capable driver framework (v4l2) and on the other hand by simplifying the task of interfacing to the user-space? I think it would help if there were more specific elaborations on the
