Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
-Original Message- From: Intel-gfx [mailto:intel-gfx-boun...@lists.freedesktop.org] On Behalf Of Imre Deak Sent: Thursday, May 22, 2014 1:08 AM On Wed, 2014-05-21 at 18:05 +0200, Daniel Vetter wrote: On Wed, May 21, 2014 at 5:56 PM, Babu, Ramesh ramesh.b...@intel.com wrote: On Tue, May 20, 2014 at 05:29:07PM +0300, Imre Deak wrote: On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. If we are able create and mandate that HDMI display controller is parent and audio is child device, then this wouldn't be an issue and PM frameowrk will ensure parent is suspended last. If there is a scenario where HDMI audio has to active but display has to go to low power, then parent-child device is not optimal. There needs to be a mechanism to turn on/off individual hw blocks within the controller. Our gfx runtime pm code is a _lot_ better than that. We track each power domain individually and enable/disable them only when need. armsoc drivers could do the same or make sure that the avsink device is a child of the right block. Of course if your driver only has binary runtime pm and fires up everything then we have a problem. But imo that's a problem with that driver, not with making avsink real devices as children of something. I would also add that at least in case of Haswell, there is really a hard dependency between the display device and the HDMI audio functionality: The power well required by HDMI is controlled via the PWR_WELL_CTL2 register which is in turn part of the display power domain. This domain is turned off when the display device is in D3 state, so to turn on audio we really have to first put the display device into D0 state. Since the PM framework doesn't provide any way to reorder the initialization of devices, we can only depend on the device parent - child relationship to achieve the above correct init order. --Imre So for Haswell, how about create a device for the 'power well' and make this power device be a child of the display device? And by any means (e.g. further extract the device as a power object and expose to audio driver), the audio driver can finally trigger pm_runtime_get/put_sync() on this power device to solve the power dependency on audio side, and the parent-child relationship will assure the order on drm side. I feel it's a natural way for HD-Audio driver, which already binds to the HD-A controller. And for MCG HDMI audio driver which directly feeds data from system memory to the display device, I think it can either use pm_runtime_get/put_sync() on this power device (seems no necessary), or just make the audio device as child of the display device. Thanks Mengdong
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Tue, May 20, 2014 at 05:29:07PM +0300, Imre Deak wrote: On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. If we are able create and mandate that HDMI display controller is parent and audio is child device, then this wouldn't be an issue and PM frameowrk will ensure parent is suspended last. If there is a scenario where HDMI audio has to active but display has to go to low power, then parent-child device is not optimal. There needs to be a mechanism to turn on/off individual hw blocks within the controller. ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Wed, May 21, 2014 at 5:56 PM, Babu, Ramesh ramesh.b...@intel.com wrote: On Tue, May 20, 2014 at 05:29:07PM +0300, Imre Deak wrote: On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. If we are able create and mandate that HDMI display controller is parent and audio is child device, then this wouldn't be an issue and PM frameowrk will ensure parent is suspended last. If there is a scenario where HDMI audio has to active but display has to go to low power, then parent-child device is not optimal. There needs to be a mechanism to turn on/off individual hw blocks within the controller. Our gfx runtime pm code is a _lot_ better than that. We track each power domain individually and enable/disable them only when need. armsoc drivers could do the same or make sure that the avsink device is a child of the right block. Of course if your driver only has binary runtime pm and fires up everything then we have a problem. But imo that's a problem with that driver, not with making avsink real devices as children of something. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Wed, 2014-05-21 at 18:05 +0200, Daniel Vetter wrote: On Wed, May 21, 2014 at 5:56 PM, Babu, Ramesh ramesh.b...@intel.com wrote: On Tue, May 20, 2014 at 05:29:07PM +0300, Imre Deak wrote: On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. If we are able create and mandate that HDMI display controller is parent and audio is child device, then this wouldn't be an issue and PM frameowrk will ensure parent is suspended last. If there is a scenario where HDMI audio has to active but display has to go to low power, then parent-child device is not optimal. There needs to be a mechanism to turn on/off individual hw blocks within the controller. Our gfx runtime pm code is a _lot_ better than that. We track each power domain individually and enable/disable them only when need. armsoc drivers could do the same or make sure that the avsink device is a child of the right block. Of course if your driver only has binary runtime pm and fires up everything then we have a problem. But imo that's a problem with that driver, not with making avsink real devices as children of something. I would also add that at least in case of Haswell, there is really a hard dependency between the display device and the HDMI audio functionality: The power well required by HDMI is controlled via the PWR_WELL_CTL2 register which is in turn part of the display power domain. This domain is turned off when the display device is in D3 state, so to turn on audio we really have to first put the display device into D0 state. Since the PM framework doesn't provide any way to reorder the initialization of devices, we can only depend on the device parent - child relationship to achieve the above correct init order. --Imre signature.asc Description: This is a digitally signed message part ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
At Tue, 20 May 2014 02:52:19 +,
Lin, Mengdong wrote:
This RFC is based on previous discussion to set up a generic communication
channel between display and audio driver and
an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial
draft and your advice would be appreciated
to improve the design.
The basic idea is to create a new avsink module and let both drm and alsa
depend on it.
This new module provides a framework and APIs for synchronization between the
display and audio driver.
Thanks, this looks like a good ground to start with.
Some comments below.
1. Display/Audio Client
The avsink core provides APIs to create, register and lookup a display/audio
client.
A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver)
can create a client, add some resources
objects (shared power wells, display outputs, and audio inputs, register ops)
to the client, and then register this
client to avisink core. The peer driver can look up a registered client by a
name or type, or both. If a client gives
a valid peer client name on registration, avsink core will bind the two
clients as peer for each other. And we
expect a display client and an audio client to be peers for each other in a
system.
int avsink_new_client ( const char *name,
int type, /* client type, display or audio */
struct module *module,
void *context,
const char *peer_name,
struct avsink_client **client_ret);
int avsink_free_client (struct avsink_client *client);
int avsink_register_client(struct avsink_client *client);
int avisink_unregister_client(int client_handle);
struct avsink_client *avsink_lookup_client(const char *name, int type);
struct avsink_client {
const char *name; /* client name */
int type; /* client type*/
void *context;
struct module *module; /* top-level module for locking */
struct avsink_client *peer; /* peer client */
/* shared power wells */
struct avsink_power_well *power_well;
int num_power_wells;
The power well is Intel-specific things. Better to use a more
generic term. (And, I'm always confused what power well disable
means :)
/* endpoints, display outputs or audio inputs */
struct avsink_endpoint * endpoint;
int num_endpints;
struct avsink_registers_ops *reg_ops; /* ops to access registers of
a client */
Use const for ops pointers in general (also other cases below).
void *private_data;
...
};
On system boots, the avsink module is loaded before the display and audio
driver module. And the display and audio
driver may be loaded on parallel.
For HD-audio HDMI, both controller and codec drivers would need the
avsink access. So, both drivers will register the own client?
* If a specific display driver (eg. i915) supports avsink, it can create a
display client, add power wells and display
outputs to the client, and then register the display client to the avsink
core. Then it may look up if there is any
audio client registered, by name or type, and may find an audio client
registered by some audio driver.
* If an audio driver supports avsink, it usually should look up a registered
display client by name or type at first,
because it may need the shared power well in GPU and check the display
outputs' name to bind the audio inputs. If
the display client is not registered yet, the audio driver can choose to
wait (maybe in a work queue) or return
-EAGAIN for a deferred probe. After the display client is found, the audio
driver can register an audio client with
the display client's name as the peer name, the avsink core will bind the
display and audio clients to each other.
There is already component framework, BTW. Can we integrate it into
avsink instead?
Open question:
If the display or audio driver is disabled by the black list, shall we
introduce a time out to avoid waiting for the
other client registered endlessly?
Yes, timeout sounds like a sensible option.
2. Shared power wells (optional)
The audio and display devices, maybe only part of them, may share a common
power well (e.g. for Intel Haswell and
Broadwell). If so, the driver that controls the power well should define a
power well object, implement the get/put ops,
and add it to its avsink client before registering the client to avsink core.
Then the peer client can look up this
power well by its name, and get/put this power well as a user.
A client can have multiple power well objects.
struct avsink_power_well {
const char *name; /* name of the power well */
void *context; /* parameter of get/put ops, maybe device pointer
for this power well */
struct
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
Adding Greg just as an fyi since we've chatted briefly about the avsink
bus. Comments below.
-Daniel
On Tue, May 20, 2014 at 02:52:19AM +, Lin, Mengdong wrote:
This RFC is based on previous discussion to set up a generic communication
channel between display and audio driver and
an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial
draft and your advice would be appreciated
to improve the design.
The basic idea is to create a new avsink module and let both drm and alsa
depend on it.
This new module provides a framework and APIs for synchronization between the
display and audio driver.
1. Display/Audio Client
The avsink core provides APIs to create, register and lookup a display/audio
client.
A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver)
can create a client, add some resources
objects (shared power wells, display outputs, and audio inputs, register ops)
to the client, and then register this
client to avisink core. The peer driver can look up a registered client by a
name or type, or both. If a client gives
a valid peer client name on registration, avsink core will bind the two
clients as peer for each other. And we
expect a display client and an audio client to be peers for each other in a
system.
int avsink_new_client ( const char *name,
int type, /* client type, display or audio */
struct module *module,
void *context,
const char *peer_name,
struct avsink_client **client_ret);
int avsink_free_client (struct avsink_client *client);
Hm, my idea was to create a new avsink bus and let vga drivers register
devices on that thing and audio drivers register as drivers. There's a bit
more work involved in creating a full-blown bus, but it has a lot of
upsides:
- Established infrastructure for matching drivers (i.e. audio drivers)
against devices (i.e. avsinks exported by gfx drivers).
- Module refcounting.
- power domain handling and well-integrated into runtime pm.
- Allows integration into componentized device framework since we're
dealing with a real struct device.
- Better decoupling between gfx and audio side since registration is done
at runtime.
- We can attach drv private date which the audio driver needs.
int avsink_register_client(struct avsink_client *client);
int avisink_unregister_client(int client_handle);
struct avsink_client *avsink_lookup_client(const char *name, int type);
struct avsink_client {
const char *name; /* client name */
int type; /* client type*/
void *context;
struct module *module; /* top-level module for locking */
struct avsink_client *peer; /* peer client */
/* shared power wells */
struct avsink_power_well *power_well;
We need to have an struct power_domain here so that we can do proper
runtime pm. But like I've said above I think we actually want a full blown
struct device.
int num_power_wells;
/* endpoints, display outputs or audio inputs */
struct avsink_endpoint * endpoint;
int num_endpints;
struct avsink_registers_ops *reg_ops; /* ops to access registers of
a client */
void *private_data;
...
};
I think you're indeed implementing a full blown bus here ;-)
avsink-client = bus devices/childern
avsink-peer = driver for all this stuff
avsink-power_well = runtime pm support for the avsink bus
avsink-reg_ops = driver bind/unbind support
On system boots, the avsink module is loaded before the display and audio
driver module. And the display and audio
driver may be loaded on parallel.
* If a specific display driver (eg. i915) supports avsink, it can create a
display client, add power wells and display
outputs to the client, and then register the display client to the avsink
core. Then it may look up if there is any
audio client registered, by name or type, and may find an audio client
registered by some audio driver.
* If an audio driver supports avsink, it usually should look up a registered
display client by name or type at first,
because it may need the shared power well in GPU and check the display
outputs' name to bind the audio inputs. If
the display client is not registered yet, the audio driver can choose to
wait (maybe in a work queue) or return
-EAGAIN for a deferred probe. After the display client is found, the audio
driver can register an audio client with
the display client's name as the peer name, the avsink core will bind the
display and audio clients to each other.
Open question:
If the display or audio driver is disabled by the black list, shall we
introduce a time out to avoid waiting for the
other client registered endlessly?
If the hdmi/dp side is a separate audio instance then we can just
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
Also adding dri-devel and linux-media. Please don't forget these lists for
the next round.
-Daniel
On Tue, May 20, 2014 at 12:02:04PM +0200, Daniel Vetter wrote:
Adding Greg just as an fyi since we've chatted briefly about the avsink
bus. Comments below.
-Daniel
On Tue, May 20, 2014 at 02:52:19AM +, Lin, Mengdong wrote:
This RFC is based on previous discussion to set up a generic communication
channel between display and audio driver and
an internal design of Intel MCG/VPG HDMI audio driver. It's still an
initial draft and your advice would be appreciated
to improve the design.
The basic idea is to create a new avsink module and let both drm and alsa
depend on it.
This new module provides a framework and APIs for synchronization between
the display and audio driver.
1. Display/Audio Client
The avsink core provides APIs to create, register and lookup a
display/audio client.
A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver)
can create a client, add some resources
objects (shared power wells, display outputs, and audio inputs, register
ops) to the client, and then register this
client to avisink core. The peer driver can look up a registered client by
a name or type, or both. If a client gives
a valid peer client name on registration, avsink core will bind the two
clients as peer for each other. And we
expect a display client and an audio client to be peers for each other in a
system.
int avsink_new_client ( const char *name,
int type, /* client type, display or audio */
struct module *module,
void *context,
const char *peer_name,
struct avsink_client **client_ret);
int avsink_free_client (struct avsink_client *client);
Hm, my idea was to create a new avsink bus and let vga drivers register
devices on that thing and audio drivers register as drivers. There's a bit
more work involved in creating a full-blown bus, but it has a lot of
upsides:
- Established infrastructure for matching drivers (i.e. audio drivers)
against devices (i.e. avsinks exported by gfx drivers).
- Module refcounting.
- power domain handling and well-integrated into runtime pm.
- Allows integration into componentized device framework since we're
dealing with a real struct device.
- Better decoupling between gfx and audio side since registration is done
at runtime.
- We can attach drv private date which the audio driver needs.
int avsink_register_client(struct avsink_client *client);
int avisink_unregister_client(int client_handle);
struct avsink_client *avsink_lookup_client(const char *name, int type);
struct avsink_client {
const char *name; /* client name */
int type; /* client type*/
void *context;
struct module *module; /* top-level module for locking */
struct avsink_client *peer; /* peer client */
/* shared power wells */
struct avsink_power_well *power_well;
We need to have an struct power_domain here so that we can do proper
runtime pm. But like I've said above I think we actually want a full blown
struct device.
int num_power_wells;
/* endpoints, display outputs or audio inputs */
struct avsink_endpoint * endpoint;
int num_endpints;
struct avsink_registers_ops *reg_ops; /* ops to access registers
of a client */
void *private_data;
...
};
I think you're indeed implementing a full blown bus here ;-)
avsink-client = bus devices/childern
avsink-peer = driver for all this stuff
avsink-power_well = runtime pm support for the avsink bus
avsink-reg_ops = driver bind/unbind support
On system boots, the avsink module is loaded before the display and audio
driver module. And the display and audio
driver may be loaded on parallel.
* If a specific display driver (eg. i915) supports avsink, it can create a
display client, add power wells and display
outputs to the client, and then register the display client to the avsink
core. Then it may look up if there is any
audio client registered, by name or type, and may find an audio client
registered by some audio driver.
* If an audio driver supports avsink, it usually should look up a
registered display client by name or type at first,
because it may need the shared power well in GPU and check the display
outputs' name to bind the audio inputs. If
the display client is not registered yet, the audio driver can choose to
wait (maybe in a work queue) or return
-EAGAIN for a deferred probe. After the display client is found, the
audio driver can register an audio client with
the display client's name as the peer name, the avsink core will bind the
display
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Tue, May 20, 2014 at 10:10:00AM +0200, Takashi Iwai wrote:
struct avsink_client {
const char *name; /* client name */
int type; /* client type*/
void *context;
struct module *module; /* top-level module for locking */
struct avsink_client *peer; /* peer client */
/* shared power wells */
struct avsink_power_well *power_well;
int num_power_wells;
The power well is Intel-specific things. Better to use a more
generic term. (And, I'm always confused what power well disable
means :)
Given that runtime pm is a prevalent usage, wouldn't it make sense
to say that I am HDMI client so keep the resources on? This can be easily
managed if we are able to create the audio device as child of display
controller. That would be implementation agnostic and controller can do whatever
(power well or not) to keep it on/off
/* endpoints, display outputs or audio inputs */
struct avsink_endpoint * endpoint;
int num_endpints;
struct avsink_registers_ops *reg_ops; /* ops to access registers
of a client */
Use const for ops pointers in general (also other cases below).
void *private_data;
...
};
On system boots, the avsink module is loaded before the display and audio
driver module. And the display and audio
driver may be loaded on parallel.
For HD-audio HDMI, both controller and codec drivers would need the
avsink access. So, both drivers will register the own client?
that sound logical here..
* If a specific display driver (eg. i915) supports avsink, it can create a
display client, add power wells and display
outputs to the client, and then register the display client to the avsink
core. Then it may look up if there is any
audio client registered, by name or type, and may find an audio client
registered by some audio driver.
* If an audio driver supports avsink, it usually should look up a
registered display client by name or type at first,
because it may need the shared power well in GPU and check the display
outputs' name to bind the audio inputs. If
the display client is not registered yet, the audio driver can choose to
wait (maybe in a work queue) or return
-EAGAIN for a deferred probe. After the display client is found, the
audio driver can register an audio client with
the display client's name as the peer name, the avsink core will bind the
display and audio clients to each other.
There is already component framework, BTW. Can we integrate it into
avsink instead?
Open question:
If the display or audio driver is disabled by the black list, shall we
introduce a time out to avoid waiting for the
other client registered endlessly?
Yes, timeout sounds like a sensible option.
2. Shared power wells (optional)
The audio and display devices, maybe only part of them, may share a common
power well (e.g. for Intel Haswell and
Broadwell). If so, the driver that controls the power well should define a
power well object, implement the get/put ops,
and add it to its avsink client before registering the client to avsink
core. Then the peer client can look up this
power well by its name, and get/put this power well as a user.
A client can have multiple power well objects.
struct avsink_power_well {
const char *name; /* name of the power well */
void *context; /* parameter of get/put ops, maybe device pointer
for this power well */
struct avsink_power_well_ops *ops
};
struct avsink_power_well_ops {
int (*get)(void *context);
int (*put)(void *context);
};
API:
int avsink_new_power(struct avsink_client *client,
const char *power_name,
void * power_context,
struct avsink_power_well_ops *ops,
struct avsink_power_well **power_ret);
struct avsink_power_well *avisnk_lookup_power(const char *name);
int avsink_get_power(struct avsink_power_well *power); /* Reqesut the
power */
int avsink_put_power(struct avsink_power_well *power);/* Release the
power */
For example, the i915 display driver can create a device for the shared
power well in Haswell GPU, implement its PM
functions, and use the device pointer as the context when creating the
power well object, like below
struct avsink_power_well_ops i915_power_well_ops = {
.get = pm_runtime_get_sync;
.put = pm_runtime_put_sync;
};
This needs function pointer cast, and it's not portable although it'd
work practically.
...
avsink_new_power ( display_client,
i915_display_power_well,
pdev, /* pointer of the power well device */
i915_power_well_ops,
...)
Power
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Tue, May 20, 2014 at 12:04:38PM +0200, Daniel Vetter wrote:
Also adding dri-devel and linux-media. Please don't forget these lists for
the next round.
-Daniel
On Tue, May 20, 2014 at 12:02:04PM +0200, Daniel Vetter wrote:
Adding Greg just as an fyi since we've chatted briefly about the avsink
bus. Comments below.
-Daniel
On Tue, May 20, 2014 at 02:52:19AM +, Lin, Mengdong wrote:
This RFC is based on previous discussion to set up a generic
communication channel between display and audio driver and
an internal design of Intel MCG/VPG HDMI audio driver. It's still an
initial draft and your advice would be appreciated
to improve the design.
The basic idea is to create a new avsink module and let both drm and alsa
depend on it.
This new module provides a framework and APIs for synchronization between
the display and audio driver.
1. Display/Audio Client
The avsink core provides APIs to create, register and lookup a
display/audio client.
A specific display driver (eg. i915) or audio driver (eg. HD-Audio
driver) can create a client, add some resources
objects (shared power wells, display outputs, and audio inputs, register
ops) to the client, and then register this
client to avisink core. The peer driver can look up a registered client
by a name or type, or both. If a client gives
a valid peer client name on registration, avsink core will bind the two
clients as peer for each other. And we
expect a display client and an audio client to be peers for each other in
a system.
int avsink_new_client ( const char *name,
int type, /* client type, display or audio
*/
struct module *module,
void *context,
const char *peer_name,
struct avsink_client **client_ret);
int avsink_free_client (struct avsink_client *client);
Hm, my idea was to create a new avsink bus and let vga drivers register
devices on that thing and audio drivers register as drivers. There's a bit
more work involved in creating a full-blown bus, but it has a lot of
upsides:
- Established infrastructure for matching drivers (i.e. audio drivers)
against devices (i.e. avsinks exported by gfx drivers).
- Module refcounting.
- power domain handling and well-integrated into runtime pm.
- Allows integration into componentized device framework since we're
dealing with a real struct device.
- Better decoupling between gfx and audio side since registration is done
at runtime.
- We can attach drv private date which the audio driver needs.
I think this would be another case where the interface framework[0]
could potentially be used. It doesn't give you all of the above, but
there's no reason it couldn't be extended. Then again, adding too much
would end up duplicating more of the driver core, so if something really
heavy-weight is required here, then the interface framework is not the
best option.
[0]: https://lkml.org/lkml/2014/5/13/525
On system boots, the avsink module is loaded before the display and audio
driver module. And the display and audio
driver may be loaded on parallel.
* If a specific display driver (eg. i915) supports avsink, it can create
a display client, add power wells and display
outputs to the client, and then register the display client to the
avsink core. Then it may look up if there is any
audio client registered, by name or type, and may find an audio client
registered by some audio driver.
* If an audio driver supports avsink, it usually should look up a
registered display client by name or type at first,
because it may need the shared power well in GPU and check the display
outputs' name to bind the audio inputs. If
the display client is not registered yet, the audio driver can choose
to wait (maybe in a work queue) or return
-EAGAIN for a deferred probe. After the display client is found, the
audio driver can register an audio client with
-EPROBE_DEFER is the correct error code for deferred probing.
6. Display register operation (optional)
Some audio driver needs to access GPU audio registers. The register ops
are provided by the peer display client.
struct avsink_registers_ops {
int (*read_register) (uint32_t reg_addr, uint32_t *data, void
*context);
int (*write_register) (uint32_t reg_addr, uint32_t data, void
*context);
int (*read_modify_register) (uint32_t reg_addr, uint32_t data,
uint32_t mask, void *context);
int avsink_define_reg_ops (struct avsink_client *client, struct
avsink_registers_ops *ops);
And avsink core provides API for the audio driver to access the display
registers:
int avsink_read_display_register(struct avsink_client *client ,
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. --Imre signature.asc Description: This is a digitally signed message part ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Tue, May 20, 2014 at 4:29 PM, Imre Deak imre.d...@intel.com wrote: On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. Yeah, this is a big reason why I want real devices - we have piles of infrastructure to solve these ordering issues as soon as there's a struct device around. If we don't use that, we need to reinvent all those wheels ourselves. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Tue, May 20, 2014 at 05:29:07PM +0300, Imre Deak wrote: On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. If we are able create and mandate that HDMI display controller is parent and audio is child device, then this wouldn't be an issue and PM frameowrk will ensure parent is suspended last. -- ~Vinod signature.asc Description: Digital signature ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Re: [Intel-gfx] [RFC] set up an sync channel between audio and display driver (i.e. ALSA and DRM)
On Tue, 2014-05-20 at 20:05 +0530, Vinod Koul wrote: On Tue, May 20, 2014 at 05:29:07PM +0300, Imre Deak wrote: On Tue, 2014-05-20 at 05:52 +0300, Lin, Mengdong wrote: This RFC is based on previous discussion to set up a generic communication channel between display and audio driver and an internal design of Intel MCG/VPG HDMI audio driver. It's still an initial draft and your advice would be appreciated to improve the design. The basic idea is to create a new avsink module and let both drm and alsa depend on it. This new module provides a framework and APIs for synchronization between the display and audio driver. 1. Display/Audio Client The avsink core provides APIs to create, register and lookup a display/audio client. A specific display driver (eg. i915) or audio driver (eg. HD-Audio driver) can create a client, add some resources objects (shared power wells, display outputs, and audio inputs, register ops) to the client, and then register this client to avisink core. The peer driver can look up a registered client by a name or type, or both. If a client gives a valid peer client name on registration, avsink core will bind the two clients as peer for each other. And we expect a display client and an audio client to be peers for each other in a system. One problem we have at the moment is the order of calling the system suspend/resume handlers of the display driver wrt. that of the audio driver. Since the power well control is part of the display HW block, we need to run the display driver's resume handler first, initialize the HW, and only then let the audio driver's resume handler run. For similar reasons we have to call the audio suspend handler first and only then the display driver resume handler. Currently we solve this using the display driver's late/early suspend/resume hooks, but we'd need a more robust solution. This seems to be a similar issue to the load time ordering problem that you describe later. Having a real device for avsync that would be a child of the display device would solve the ordering issue in both cases. I admit I haven't looked into it if this is feasible, but I would like to see some solution to this as part of the plan. If we are able create and mandate that HDMI display controller is parent and audio is child device, then this wouldn't be an issue and PM frameowrk will ensure parent is suspended last. To my understanding we can't really do that since that's already fixed by the physical bus topology. That is in the Intel case the parent of both the audio and display device is the corresponding PCI bridge device. But avsync could be a new virtual device and you could let that be the child of the display device. --Imre signature.asc Description: This is a digitally signed message part ___ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/intel-gfx
