Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
On 21/02/2024 11:11, Shengjiu Wang wrote:
> On Wed, Feb 21, 2024 at 12:30 PM Tomasz Figa wrote:
>>
>> On Sat, Feb 17, 2024 at 6:42 PM Mauro Carvalho Chehab
>> wrote:
>>>
>>> Em Thu, 18 Jan 2024 20:32:00 +0800
>>> Shengjiu Wang escreveu:
>>>
Audio signal processing has the requirement for memory to
memory similar as Video.
This patch is to add this support in v4l2 framework, defined
new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
for audio case usage.
The created audio device is named "/dev/v4l-audioX".
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/buffer.rst| 6 ++
.../media/v4l/dev-audio-mem2mem.rst | 71 +++
.../userspace-api/media/v4l/devices.rst | 1 +
.../media/v4l/vidioc-enum-fmt.rst | 2 +
.../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
.../media/videodev2.h.rst.exceptions | 2 +
.../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
drivers/media/v4l2-core/v4l2-dev.c| 17 +
drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
include/media/v4l2-dev.h | 2 +
include/media/v4l2-ioctl.h| 34 +
include/uapi/linux/videodev2.h| 17 +
13 files changed, 222 insertions(+)
create mode 100644
Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
b/Documentation/userspace-api/media/v4l/buffer.rst
index 52bbee81c080..a3754ca6f0d6 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst
@@ -438,6 +438,12 @@ enum v4l2_buf_type
* - ``V4L2_BUF_TYPE_META_OUTPUT``
- 14
>>>
- Buffer for metadata output, see :ref:`metadata`.
+* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
+ - 15
+ - Buffer for audio capture, see :ref:`audio`.
+* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
+ - 16
>>>
>>> Hmm... alsa APi define input/output as:
>>> enum {
>>> SNDRV_PCM_STREAM_PLAYBACK = 0,
>>> SNDRV_PCM_STREAM_CAPTURE,
>>> SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
>>> };
>>>
>>>
>>> I would use a namespace as close as possible to the
>>> ALSA API. Also, we're not talking about V4L2, but, instead
>>> audio. so, not sure if I like the prefix to start with
>>> V4L2_. Maybe ALSA_?
>>>
>>> So, a better namespace would be:
>>>
>>> ${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
>>> and
>>> ${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
>>>
>>
>> The API is still V4L2, and all the other non-video buf types also use
>> the V4L2_ prefix, so perhaps that's good here as well?
>>
>> Whether AUDIO or PCM_STREAM makes more sense goes outside of my
>> expertise. Subjectively, a PCM stream sounds more specific than an
>> audio stream. Do those buf types also support non-PCM audio streams?
>
> Currently I use it for PCM, but I think it can also be used for non-PCM.
> So use the below name?
> V4L2_BUF_TYPE_AUDIO_CAPTURE
> V4L2_BUF_TYPE_AUDIO_PLAYBACK
I really prefer keeping the names as they are in this patch. CAPTURE/OUTPUT
is consistent with V4L2 nomenclature, and since this is a M2M device 'PLAYBACK'
isn't really a good name either. It's not an audio playback device, it's a
rate converter.
Regards,
Hans
>
>>
+ - Buffer for audio output, see :ref:`audio`.
.. _buffer-flags:
diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
new file mode 100644
index ..68faecfe3a02
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
@@ -0,0 +1,71 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiomem2mem:
+
+
+Audio Memory-To-Memory Interface
+
+
+An audio memory-to-memory device can compress, decompress, transform, or
+otherwise convert audio data from one format into another format, in
memory.
+Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
+Examples of memory-to-memory devices are audio codecs, audio
preprocessing,
+audio postprocessing.
+
+A memory-to-memory audio node supports both output (sending audio frames
from
+memory to the hardware) and capture (receiving the processed audio frames
+from the hardware into memory) stream I/O. An application will have to
+setup the stream I/O for both sides and finally call
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
On 17/02/2024 10:42, Mauro Carvalho Chehab wrote:
> Em Thu, 18 Jan 2024 20:32:00 +0800
> Shengjiu Wang escreveu:
>
>> Audio signal processing has the requirement for memory to
>> memory similar as Video.
>>
>> This patch is to add this support in v4l2 framework, defined
>> new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
>> V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
>> for audio case usage.
>>
>> The created audio device is named "/dev/v4l-audioX".
>>
>> Signed-off-by: Shengjiu Wang
>> ---
>> .../userspace-api/media/v4l/buffer.rst| 6 ++
>> .../media/v4l/dev-audio-mem2mem.rst | 71 +++
>> .../userspace-api/media/v4l/devices.rst | 1 +
>> .../media/v4l/vidioc-enum-fmt.rst | 2 +
>> .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
>> .../media/videodev2.h.rst.exceptions | 2 +
>> .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
>> drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
>> drivers/media/v4l2-core/v4l2-dev.c| 17 +
>> drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
>> include/media/v4l2-dev.h | 2 +
>> include/media/v4l2-ioctl.h| 34 +
>> include/uapi/linux/videodev2.h| 17 +
>> 13 files changed, 222 insertions(+)
>> create mode 100644
>> Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
>>
>> diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
>> b/Documentation/userspace-api/media/v4l/buffer.rst
>> index 52bbee81c080..a3754ca6f0d6 100644
>> --- a/Documentation/userspace-api/media/v4l/buffer.rst
>> +++ b/Documentation/userspace-api/media/v4l/buffer.rst
>> @@ -438,6 +438,12 @@ enum v4l2_buf_type
>> * - ``V4L2_BUF_TYPE_META_OUTPUT``
>>- 14
>
>>- Buffer for metadata output, see :ref:`metadata`.
>> +* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
>> + - 15
>> + - Buffer for audio capture, see :ref:`audio`.
>> +* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
>> + - 16
>
> Hmm... alsa APi define input/output as:
> enum {
> SNDRV_PCM_STREAM_PLAYBACK = 0,
> SNDRV_PCM_STREAM_CAPTURE,
> SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
> };
>
>
> I would use a namespace as close as possible to the
> ALSA API. Also, we're not talking about V4L2, but, instead
> audio. so, not sure if I like the prefix to start with
> V4L2_. Maybe ALSA_?
>
> So, a better namespace would be:
>
> ${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
> and
> ${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
>
>> + - Buffer for audio output, see :ref:`audio`.
>>
>>
>> .. _buffer-flags:
>> diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
>> b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
>> new file mode 100644
>> index ..68faecfe3a02
>> --- /dev/null
>> +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
>> @@ -0,0 +1,71 @@
>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>> +
>> +.. _audiomem2mem:
>> +
>> +
>> +Audio Memory-To-Memory Interface
>> +
>> +
>> +An audio memory-to-memory device can compress, decompress, transform, or
>> +otherwise convert audio data from one format into another format, in memory.
>> +Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
>> +Examples of memory-to-memory devices are audio codecs, audio preprocessing,
>> +audio postprocessing.
>> +
>> +A memory-to-memory audio node supports both output (sending audio frames
>> from
>> +memory to the hardware) and capture (receiving the processed audio frames
>> +from the hardware into memory) stream I/O. An application will have to
>> +setup the stream I/O for both sides and finally call
>> +:ref:`VIDIOC_STREAMON ` for both capture and output to
>> +start the hardware.
>> +
>> +Memory-to-memory devices function as a shared resource: you can
>> +open the audio node multiple times, each application setting up their
>> +own properties that are local to the file handle, and each can use
>> +it independently from the others. The driver will arbitrate access to
>> +the hardware and reprogram it whenever another file handler gets access.
>> +
>> +Audio memory-to-memory devices are accessed through character device
>> +special files named ``/dev/v4l-audio``
>> +
>> +Querying Capabilities
>> +=
>> +
>> +Device nodes supporting the audio memory-to-memory interface set the
>> +``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
>> +:c:type:`v4l2_capability` structure returned by the
>> :c:func:`VIDIOC_QUERYCAP`
>> +ioctl.
>> +
>> +Data Format Negotiation
>> +===
>> +
>> +The audio device uses the :ref:`format` ioctls to select the capture format.
>> +The audio buffer content format is bound to that selected format. In
>> addition
>> +to the basic
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
On Wed, Feb 21, 2024 at 12:30 PM Tomasz Figa wrote:
>
> On Sat, Feb 17, 2024 at 6:42 PM Mauro Carvalho Chehab
> wrote:
> >
> > Em Thu, 18 Jan 2024 20:32:00 +0800
> > Shengjiu Wang escreveu:
> >
> > > Audio signal processing has the requirement for memory to
> > > memory similar as Video.
> > >
> > > This patch is to add this support in v4l2 framework, defined
> > > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
> > > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
> > > for audio case usage.
> > >
> > > The created audio device is named "/dev/v4l-audioX".
> > >
> > > Signed-off-by: Shengjiu Wang
> > > ---
> > > .../userspace-api/media/v4l/buffer.rst| 6 ++
> > > .../media/v4l/dev-audio-mem2mem.rst | 71 +++
> > > .../userspace-api/media/v4l/devices.rst | 1 +
> > > .../media/v4l/vidioc-enum-fmt.rst | 2 +
> > > .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
> > > .../media/videodev2.h.rst.exceptions | 2 +
> > > .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
> > > drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
> > > drivers/media/v4l2-core/v4l2-dev.c| 17 +
> > > drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
> > > include/media/v4l2-dev.h | 2 +
> > > include/media/v4l2-ioctl.h| 34 +
> > > include/uapi/linux/videodev2.h| 17 +
> > > 13 files changed, 222 insertions(+)
> > > create mode 100644
> > > Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > >
> > > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
> > > b/Documentation/userspace-api/media/v4l/buffer.rst
> > > index 52bbee81c080..a3754ca6f0d6 100644
> > > --- a/Documentation/userspace-api/media/v4l/buffer.rst
> > > +++ b/Documentation/userspace-api/media/v4l/buffer.rst
> > > @@ -438,6 +438,12 @@ enum v4l2_buf_type
> > > * - ``V4L2_BUF_TYPE_META_OUTPUT``
> > >- 14
> >
> > >- Buffer for metadata output, see :ref:`metadata`.
> > > +* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
> > > + - 15
> > > + - Buffer for audio capture, see :ref:`audio`.
> > > +* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
> > > + - 16
> >
> > Hmm... alsa APi define input/output as:
> > enum {
> > SNDRV_PCM_STREAM_PLAYBACK = 0,
> > SNDRV_PCM_STREAM_CAPTURE,
> > SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
> > };
> >
> >
> > I would use a namespace as close as possible to the
> > ALSA API. Also, we're not talking about V4L2, but, instead
> > audio. so, not sure if I like the prefix to start with
> > V4L2_. Maybe ALSA_?
> >
> > So, a better namespace would be:
> >
> > ${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
> > and
> > ${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
> >
>
> The API is still V4L2, and all the other non-video buf types also use
> the V4L2_ prefix, so perhaps that's good here as well?
>
> Whether AUDIO or PCM_STREAM makes more sense goes outside of my
> expertise. Subjectively, a PCM stream sounds more specific than an
> audio stream. Do those buf types also support non-PCM audio streams?
Currently I use it for PCM, but I think it can also be used for non-PCM.
So use the below name?
V4L2_BUF_TYPE_AUDIO_CAPTURE
V4L2_BUF_TYPE_AUDIO_PLAYBACK
>
> > > + - Buffer for audio output, see :ref:`audio`.
> > >
> > >
> > > .. _buffer-flags:
> > > diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > > b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > > new file mode 100644
> > > index ..68faecfe3a02
> > > --- /dev/null
> > > +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > > @@ -0,0 +1,71 @@
> > > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > > +
> > > +.. _audiomem2mem:
> > > +
> > > +
> > > +Audio Memory-To-Memory Interface
> > > +
> > > +
> > > +An audio memory-to-memory device can compress, decompress, transform, or
> > > +otherwise convert audio data from one format into another format, in
> > > memory.
> > > +Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
> > > +Examples of memory-to-memory devices are audio codecs, audio
> > > preprocessing,
> > > +audio postprocessing.
> > > +
> > > +A memory-to-memory audio node supports both output (sending audio frames
> > > from
> > > +memory to the hardware) and capture (receiving the processed audio frames
> > > +from the hardware into memory) stream I/O. An application will have to
> > > +setup the stream I/O for both sides and finally call
> > > +:ref:`VIDIOC_STREAMON ` for both capture and output to
> > > +start the hardware.
> > > +
> > > +Memory-to-memory devices function as a shared resource: you can
> > > +open the audio node multiple times, each application setting up their
> > > +own properties
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
On Thu, Jan 18, 2024 at 10:15 PM Shengjiu Wang wrote: > > Audio signal processing has the requirement for memory to > memory similar as Video. > > This patch is to add this support in v4l2 framework, defined > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format > for audio case usage. > > The created audio device is named "/dev/v4l-audioX". > > Signed-off-by: Shengjiu Wang > --- > .../userspace-api/media/v4l/buffer.rst| 6 ++ > .../media/v4l/dev-audio-mem2mem.rst | 71 +++ > .../userspace-api/media/v4l/devices.rst | 1 + > .../media/v4l/vidioc-enum-fmt.rst | 2 + > .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++ > .../media/videodev2.h.rst.exceptions | 2 + > .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++ > drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++ > drivers/media/v4l2-core/v4l2-dev.c| 17 + > drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++ > include/media/v4l2-dev.h | 2 + > include/media/v4l2-ioctl.h| 34 + > include/uapi/linux/videodev2.h| 17 + > 13 files changed, 222 insertions(+) > create mode 100644 > Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst For drivers/media/common/videobuf2: Acked-by: Tomasz Figa Best regards, Tomasz
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
On Sat, Feb 17, 2024 at 6:42 PM Mauro Carvalho Chehab
wrote:
>
> Em Thu, 18 Jan 2024 20:32:00 +0800
> Shengjiu Wang escreveu:
>
> > Audio signal processing has the requirement for memory to
> > memory similar as Video.
> >
> > This patch is to add this support in v4l2 framework, defined
> > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
> > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
> > for audio case usage.
> >
> > The created audio device is named "/dev/v4l-audioX".
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > .../userspace-api/media/v4l/buffer.rst| 6 ++
> > .../media/v4l/dev-audio-mem2mem.rst | 71 +++
> > .../userspace-api/media/v4l/devices.rst | 1 +
> > .../media/v4l/vidioc-enum-fmt.rst | 2 +
> > .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
> > .../media/videodev2.h.rst.exceptions | 2 +
> > .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
> > drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
> > drivers/media/v4l2-core/v4l2-dev.c| 17 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
> > include/media/v4l2-dev.h | 2 +
> > include/media/v4l2-ioctl.h| 34 +
> > include/uapi/linux/videodev2.h| 17 +
> > 13 files changed, 222 insertions(+)
> > create mode 100644
> > Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
> > b/Documentation/userspace-api/media/v4l/buffer.rst
> > index 52bbee81c080..a3754ca6f0d6 100644
> > --- a/Documentation/userspace-api/media/v4l/buffer.rst
> > +++ b/Documentation/userspace-api/media/v4l/buffer.rst
> > @@ -438,6 +438,12 @@ enum v4l2_buf_type
> > * - ``V4L2_BUF_TYPE_META_OUTPUT``
> >- 14
>
> >- Buffer for metadata output, see :ref:`metadata`.
> > +* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
> > + - 15
> > + - Buffer for audio capture, see :ref:`audio`.
> > +* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
> > + - 16
>
> Hmm... alsa APi define input/output as:
> enum {
> SNDRV_PCM_STREAM_PLAYBACK = 0,
> SNDRV_PCM_STREAM_CAPTURE,
> SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
> };
>
>
> I would use a namespace as close as possible to the
> ALSA API. Also, we're not talking about V4L2, but, instead
> audio. so, not sure if I like the prefix to start with
> V4L2_. Maybe ALSA_?
>
> So, a better namespace would be:
>
> ${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
> and
> ${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
>
The API is still V4L2, and all the other non-video buf types also use
the V4L2_ prefix, so perhaps that's good here as well?
Whether AUDIO or PCM_STREAM makes more sense goes outside of my
expertise. Subjectively, a PCM stream sounds more specific than an
audio stream. Do those buf types also support non-PCM audio streams?
> > + - Buffer for audio output, see :ref:`audio`.
> >
> >
> > .. _buffer-flags:
> > diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > new file mode 100644
> > index ..68faecfe3a02
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > @@ -0,0 +1,71 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _audiomem2mem:
> > +
> > +
> > +Audio Memory-To-Memory Interface
> > +
> > +
> > +An audio memory-to-memory device can compress, decompress, transform, or
> > +otherwise convert audio data from one format into another format, in
> > memory.
> > +Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
> > +Examples of memory-to-memory devices are audio codecs, audio preprocessing,
> > +audio postprocessing.
> > +
> > +A memory-to-memory audio node supports both output (sending audio frames
> > from
> > +memory to the hardware) and capture (receiving the processed audio frames
> > +from the hardware into memory) stream I/O. An application will have to
> > +setup the stream I/O for both sides and finally call
> > +:ref:`VIDIOC_STREAMON ` for both capture and output to
> > +start the hardware.
> > +
> > +Memory-to-memory devices function as a shared resource: you can
> > +open the audio node multiple times, each application setting up their
> > +own properties that are local to the file handle, and each can use
> > +it independently from the others. The driver will arbitrate access to
> > +the hardware and reprogram it whenever another file handler gets access.
> > +
> > +Audio memory-to-memory devices are accessed through character device
> > +special files named ``/dev/v4l-audio``
> > +
> > +Querying Capabilities
> > +=
> > +
> > +Device nodes supporting the audio
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
Hi Mauro, Hans
On Sat, Feb 17, 2024 at 5:42 PM Mauro Carvalho Chehab
wrote:
>
> Em Thu, 18 Jan 2024 20:32:00 +0800
> Shengjiu Wang escreveu:
>
> > Audio signal processing has the requirement for memory to
> > memory similar as Video.
> >
> > This patch is to add this support in v4l2 framework, defined
> > new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
> > V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
> > for audio case usage.
> >
> > The created audio device is named "/dev/v4l-audioX".
> >
> > Signed-off-by: Shengjiu Wang
> > ---
> > .../userspace-api/media/v4l/buffer.rst| 6 ++
> > .../media/v4l/dev-audio-mem2mem.rst | 71 +++
> > .../userspace-api/media/v4l/devices.rst | 1 +
> > .../media/v4l/vidioc-enum-fmt.rst | 2 +
> > .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
> > .../media/videodev2.h.rst.exceptions | 2 +
> > .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
> > drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
> > drivers/media/v4l2-core/v4l2-dev.c| 17 +
> > drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
> > include/media/v4l2-dev.h | 2 +
> > include/media/v4l2-ioctl.h| 34 +
> > include/uapi/linux/videodev2.h| 17 +
> > 13 files changed, 222 insertions(+)
> > create mode 100644
> > Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
> > b/Documentation/userspace-api/media/v4l/buffer.rst
> > index 52bbee81c080..a3754ca6f0d6 100644
> > --- a/Documentation/userspace-api/media/v4l/buffer.rst
> > +++ b/Documentation/userspace-api/media/v4l/buffer.rst
> > @@ -438,6 +438,12 @@ enum v4l2_buf_type
> > * - ``V4L2_BUF_TYPE_META_OUTPUT``
> >- 14
>
> >- Buffer for metadata output, see :ref:`metadata`.
> > +* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
> > + - 15
> > + - Buffer for audio capture, see :ref:`audio`.
> > +* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
> > + - 16
>
> Hmm... alsa APi define input/output as:
> enum {
> SNDRV_PCM_STREAM_PLAYBACK = 0,
> SNDRV_PCM_STREAM_CAPTURE,
> SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
> };
>
>
> I would use a namespace as close as possible to the
> ALSA API. Also, we're not talking about V4L2, but, instead
> audio. so, not sure if I like the prefix to start with
> V4L2_. Maybe ALSA_?
>
> So, a better namespace would be:
>
> ${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
> and
> ${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
>
Ok, So can I use the name below?
V4L2_BUF_TYPE_PCM_STREAM_PLAYBACK
V4L2_BUF_TYPE_PCM_STREAM_CAPTURE
> > + - Buffer for audio output, see :ref:`audio`.
> >
> >
> > .. _buffer-flags:
> > diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > new file mode 100644
> > index ..68faecfe3a02
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> > @@ -0,0 +1,71 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _audiomem2mem:
> > +
> > +
> > +Audio Memory-To-Memory Interface
> > +
> > +
> > +An audio memory-to-memory device can compress, decompress, transform, or
> > +otherwise convert audio data from one format into another format, in
> > memory.
> > +Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
> > +Examples of memory-to-memory devices are audio codecs, audio preprocessing,
> > +audio postprocessing.
> > +
> > +A memory-to-memory audio node supports both output (sending audio frames
> > from
> > +memory to the hardware) and capture (receiving the processed audio frames
> > +from the hardware into memory) stream I/O. An application will have to
> > +setup the stream I/O for both sides and finally call
> > +:ref:`VIDIOC_STREAMON ` for both capture and output to
> > +start the hardware.
> > +
> > +Memory-to-memory devices function as a shared resource: you can
> > +open the audio node multiple times, each application setting up their
> > +own properties that are local to the file handle, and each can use
> > +it independently from the others. The driver will arbitrate access to
> > +the hardware and reprogram it whenever another file handler gets access.
> > +
> > +Audio memory-to-memory devices are accessed through character device
> > +special files named ``/dev/v4l-audio``
> > +
> > +Querying Capabilities
> > +=
> > +
> > +Device nodes supporting the audio memory-to-memory interface set the
> > +``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
> > +:c:type:`v4l2_capability` structure returned by the
> > :c:func:`VIDIOC_QUERYCAP`
> > +ioctl.
> > +
> > +Data
Re: [PATCH v12 07/15] media: v4l2: Add audio capture and output support
Em Thu, 18 Jan 2024 20:32:00 +0800
Shengjiu Wang escreveu:
> Audio signal processing has the requirement for memory to
> memory similar as Video.
>
> This patch is to add this support in v4l2 framework, defined
> new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
> V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
> for audio case usage.
>
> The created audio device is named "/dev/v4l-audioX".
>
> Signed-off-by: Shengjiu Wang
> ---
> .../userspace-api/media/v4l/buffer.rst| 6 ++
> .../media/v4l/dev-audio-mem2mem.rst | 71 +++
> .../userspace-api/media/v4l/devices.rst | 1 +
> .../media/v4l/vidioc-enum-fmt.rst | 2 +
> .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
> .../media/videodev2.h.rst.exceptions | 2 +
> .../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
> drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
> drivers/media/v4l2-core/v4l2-dev.c| 17 +
> drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
> include/media/v4l2-dev.h | 2 +
> include/media/v4l2-ioctl.h| 34 +
> include/uapi/linux/videodev2.h| 17 +
> 13 files changed, 222 insertions(+)
> create mode 100644
> Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
>
> diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
> b/Documentation/userspace-api/media/v4l/buffer.rst
> index 52bbee81c080..a3754ca6f0d6 100644
> --- a/Documentation/userspace-api/media/v4l/buffer.rst
> +++ b/Documentation/userspace-api/media/v4l/buffer.rst
> @@ -438,6 +438,12 @@ enum v4l2_buf_type
> * - ``V4L2_BUF_TYPE_META_OUTPUT``
>- 14
>- Buffer for metadata output, see :ref:`metadata`.
> +* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
> + - 15
> + - Buffer for audio capture, see :ref:`audio`.
> +* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
> + - 16
Hmm... alsa APi define input/output as:
enum {
SNDRV_PCM_STREAM_PLAYBACK = 0,
SNDRV_PCM_STREAM_CAPTURE,
SNDRV_PCM_STREAM_LAST = SNDRV_PCM_STREAM_CAPTURE,
};
I would use a namespace as close as possible to the
ALSA API. Also, we're not talking about V4L2, but, instead
audio. so, not sure if I like the prefix to start with
V4L2_. Maybe ALSA_?
So, a better namespace would be:
${prefix}_BUF_TYPE_PCM_STREAM_PLAYBACK
and
${prefix}_BUF_TYPE_PCM_STREAM_CAPTURE
> + - Buffer for audio output, see :ref:`audio`.
>
>
> .. _buffer-flags:
> diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> new file mode 100644
> index ..68faecfe3a02
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
> @@ -0,0 +1,71 @@
> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> +
> +.. _audiomem2mem:
> +
> +
> +Audio Memory-To-Memory Interface
> +
> +
> +An audio memory-to-memory device can compress, decompress, transform, or
> +otherwise convert audio data from one format into another format, in memory.
> +Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
> +Examples of memory-to-memory devices are audio codecs, audio preprocessing,
> +audio postprocessing.
> +
> +A memory-to-memory audio node supports both output (sending audio frames from
> +memory to the hardware) and capture (receiving the processed audio frames
> +from the hardware into memory) stream I/O. An application will have to
> +setup the stream I/O for both sides and finally call
> +:ref:`VIDIOC_STREAMON ` for both capture and output to
> +start the hardware.
> +
> +Memory-to-memory devices function as a shared resource: you can
> +open the audio node multiple times, each application setting up their
> +own properties that are local to the file handle, and each can use
> +it independently from the others. The driver will arbitrate access to
> +the hardware and reprogram it whenever another file handler gets access.
> +
> +Audio memory-to-memory devices are accessed through character device
> +special files named ``/dev/v4l-audio``
> +
> +Querying Capabilities
> +=
> +
> +Device nodes supporting the audio memory-to-memory interface set the
> +``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
> +:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
> +ioctl.
> +
> +Data Format Negotiation
> +===
> +
> +The audio device uses the :ref:`format` ioctls to select the capture format.
> +The audio buffer content format is bound to that selected format. In addition
> +to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must
> be
> +supported as well.
> +
> +To use the :ref:`format` ioctls applications set the ``type`` field of the
>
[PATCH v12 07/15] media: v4l2: Add audio capture and output support
Audio signal processing has the requirement for memory to
memory similar as Video.
This patch is to add this support in v4l2 framework, defined
new buffer type V4L2_BUF_TYPE_AUDIO_CAPTURE and
V4L2_BUF_TYPE_AUDIO_OUTPUT, defined new format v4l2_audio_format
for audio case usage.
The created audio device is named "/dev/v4l-audioX".
Signed-off-by: Shengjiu Wang
---
.../userspace-api/media/v4l/buffer.rst| 6 ++
.../media/v4l/dev-audio-mem2mem.rst | 71 +++
.../userspace-api/media/v4l/devices.rst | 1 +
.../media/v4l/vidioc-enum-fmt.rst | 2 +
.../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 ++
.../media/videodev2.h.rst.exceptions | 2 +
.../media/common/videobuf2/videobuf2-v4l2.c | 4 ++
drivers/media/v4l2-core/v4l2-compat-ioctl32.c | 9 +++
drivers/media/v4l2-core/v4l2-dev.c| 17 +
drivers/media/v4l2-core/v4l2-ioctl.c | 53 ++
include/media/v4l2-dev.h | 2 +
include/media/v4l2-ioctl.h| 34 +
include/uapi/linux/videodev2.h| 17 +
13 files changed, 222 insertions(+)
create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
diff --git a/Documentation/userspace-api/media/v4l/buffer.rst
b/Documentation/userspace-api/media/v4l/buffer.rst
index 52bbee81c080..a3754ca6f0d6 100644
--- a/Documentation/userspace-api/media/v4l/buffer.rst
+++ b/Documentation/userspace-api/media/v4l/buffer.rst
@@ -438,6 +438,12 @@ enum v4l2_buf_type
* - ``V4L2_BUF_TYPE_META_OUTPUT``
- 14
- Buffer for metadata output, see :ref:`metadata`.
+* - ``V4L2_BUF_TYPE_AUDIO_CAPTURE``
+ - 15
+ - Buffer for audio capture, see :ref:`audio`.
+* - ``V4L2_BUF_TYPE_AUDIO_OUTPUT``
+ - 16
+ - Buffer for audio output, see :ref:`audio`.
.. _buffer-flags:
diff --git a/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
new file mode 100644
index ..68faecfe3a02
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
@@ -0,0 +1,71 @@
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _audiomem2mem:
+
+
+Audio Memory-To-Memory Interface
+
+
+An audio memory-to-memory device can compress, decompress, transform, or
+otherwise convert audio data from one format into another format, in memory.
+Such memory-to-memory devices set the ``V4L2_CAP_AUDIO_M2M`` capability.
+Examples of memory-to-memory devices are audio codecs, audio preprocessing,
+audio postprocessing.
+
+A memory-to-memory audio node supports both output (sending audio frames from
+memory to the hardware) and capture (receiving the processed audio frames
+from the hardware into memory) stream I/O. An application will have to
+setup the stream I/O for both sides and finally call
+:ref:`VIDIOC_STREAMON ` for both capture and output to
+start the hardware.
+
+Memory-to-memory devices function as a shared resource: you can
+open the audio node multiple times, each application setting up their
+own properties that are local to the file handle, and each can use
+it independently from the others. The driver will arbitrate access to
+the hardware and reprogram it whenever another file handler gets access.
+
+Audio memory-to-memory devices are accessed through character device
+special files named ``/dev/v4l-audio``
+
+Querying Capabilities
+=
+
+Device nodes supporting the audio memory-to-memory interface set the
+``V4L2_CAP_AUDIO_M2M`` flag in the ``device_caps`` field of the
+:c:type:`v4l2_capability` structure returned by the :c:func:`VIDIOC_QUERYCAP`
+ioctl.
+
+Data Format Negotiation
+===
+
+The audio device uses the :ref:`format` ioctls to select the capture format.
+The audio buffer content format is bound to that selected format. In addition
+to the basic :ref:`format` ioctls, the :c:func:`VIDIOC_ENUM_FMT` ioctl must be
+supported as well.
+
+To use the :ref:`format` ioctls applications set the ``type`` field of the
+:c:type:`v4l2_format` structure to ``V4L2_BUF_TYPE_AUDIO_CAPTURE`` or to
+``V4L2_BUF_TYPE_AUDIO_OUTPUT``. Both drivers and applications must set the
+remainder of the :c:type:`v4l2_format` structure to 0.
+
+.. c:type:: v4l2_audio_format
+
+.. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}|
+
+.. flat-table:: struct v4l2_audio_format
+:header-rows: 0
+:stub-columns: 0
+:widths: 1 1 2
+
+* - __u32
+ - ``pixelformat``
+ - The sample format, set by the application. see :ref:`pixfmt-audio`
+* - __u32
+ - ``channels``
+ - The channel number, set by the application. channel number range is
+[1, 32].
+* - __u32
+ - ``buffersize``
+ - Maximum buffer size in bytes required for data. The value is set by the
+driver.
diff --git
