Re: [whatwg] need a way to set output format from StreamRecorder

[Nils Dagsson Moskopp]

Kevin Marks  schrieb am Tue, 15 Feb 2011 00:23:00
-0800:

> To be fair to Safari it supports far more audio formats than other
> browsers,

Yes, but it supports *other* formats than other browsers. That's the
crux of the issue. To creators of multimedia files and web developers
alike, having to provide even two formats is a nuisance.

> as it incorporates QuickTime's engine, which was designed
> to cope with multiple audio, video and file formats via well designed
> abstractions, and the Component Manager, which has lasted since the
> late 1980s itself (first public release of QuickTime was 1991, but
> the codebase goes back pre-1990).

While the achievements of quicktime engineers may certainly be
impressive, they are entirely irrelevant to the issue at hand. In a
territory where we need a least common denominator there are no
bragging rights for “the most supported formats” (especially, when the
only one every other reasonably modern browser can play isn't among all
those).

> Yes, MP3 is the de facto standard, and somehow Mozilla and Webkit
> Android still won't play them. (Bizarrely, Webkit Android 'supports' HTML5
>  without supporting any codecs or file formats at all).

AFAIK, there are patent assertions over MP3. So, while a technical
implementation may be a relatively simple task, licensing issues
may present an insurmountable obstacle to implementors not able to pay
the fees. (I really thought this was common knowledge by now.)

> Supporting playback of uncompressed audio (and uLaw, aLaw, PCM)
> in .au .aif .wav and .mov should be trivial, and not encumbered by
> any patents. Picking one to record in by default should be something
> we could agree on - which is most widely supported at the moment? WAV?

Enjoy your saturation of bandwith. While WAV may be interesting for
local audio manipulation and okay for small sound effects, it isn't a
suitable network interchange format.

-- 
Nils Dagsson Moskopp // erlehmann

Re: [whatwg] need a way to set output format from StreamRecorder

[Kevin Marks]

On Mon, Feb 14, 2011 at 11:52 PM, Nils Dagsson Moskopp <
n...@dieweltistgarnichtso.net> wrote:

> Kevin Marks  schrieb am Mon, 14 Feb 2011 22:33:13
> -0800:
>
> > On Mon, Feb 14, 2011 at 2:39 PM, Ian Hickson  wrote:
> >
> > > […]
> > >
> > > I haven't added anything here yet, mostly because I've no idea what
> > > to add. The ideal situation here is that we have one codec that
> > > everyone can read and write and so don't need anything, but that
> > > may be hopelessly optimistic.
> >
> >
> > That isn't the ideal, as it locks us into the current state of the art
> > forever. The ideal is to enable multiple codecs +formats that can be
> > swapped out over time.
>
> Yeah, because that really worked well with , the generic
> container element. Only it didn't, and with today's media elements
> people are venting stuff like “I had to encode all my sound files in
> Ogg Vorbis and MP3, just because of you, Safari. You make my life
> unnecessarily difficult.”
>
> — 
>
> As someone who sometimes produces audio (and may want to use of
> browser-provided facilities, once they become available), I may have a
> reasonable interest in interoperability between browsers.
>

To be fair to Safari it supports far more audio formats than other browsers,
as it incorporates QuickTime's engine, which was designed to cope with
multiple audio, video and file formats via well designed abstractions, and
the Component Manager, which has lasted since the late 1980s itself (first
public release of QuickTime was 1991, but the codebase goes back pre-1990).


>
> (The Enrichment Center once again reminds you that codec hell is a real
> place where your web application will be sent at the first sign of
> defiance.)
>
> On further thought, “state of the art lock-in” may not be as bad as you
> might fear: First, bandwidth and storage space are becoming cheaper
> over time; second, there is something as “good enough”, GZIP / DEFLATE
> or MP3 being such examples that serve us for over 15 years each — even
> though better specifications (Vorbis, 7z) clearly exist.
>
> Yes, MP3 is the de facto standard, and somehow Mozilla and Webkit Android
still won't play them. (Bizarrely, Webkit Android 'supports' HTML5 
without supporting any codecs or file formats at all).



> Even CPIO is used by my modern desktop system and that was defined
> around the mid-80ies (or so Wikipedia tells me, I certainly wasn't
> released at that point).
>

 and AIFF, MOV, WAV, AVI and MP4 are all based on IFF
http://en.wikipedia.org/wiki/Interchange_File_Format - a future-proof binary
chunk format form the mid 80s too.

Supporting playback of uncompressed audio (and uLaw, aLaw, PCM) in .au .aif
.wav and .mov should be trivial, and not encumbered by any patents. Picking
one to record in by default should be something we could agree on - which is
most widely supported at the moment? WAV?

Re: [whatwg] need a way to set output format from StreamRecorder

[Nils Dagsson Moskopp]

Kevin Marks  schrieb am Mon, 14 Feb 2011 22:33:13
-0800:

> On Mon, Feb 14, 2011 at 2:39 PM, Ian Hickson  wrote:
> 
> > […]
> >
> > I haven't added anything here yet, mostly because I've no idea what
> > to add. The ideal situation here is that we have one codec that
> > everyone can read and write and so don't need anything, but that
> > may be hopelessly optimistic.
> 
> 
> That isn't the ideal, as it locks us into the current state of the art
> forever. The ideal is to enable multiple codecs +formats that can be
> swapped out over time.

Yeah, because that really worked well with , the generic
container element. Only it didn't, and with today's media elements
people are venting stuff like “I had to encode all my sound files in
Ogg Vorbis and MP3, just because of you, Safari. You make my life
unnecessarily difficult.”

— 

As someone who sometimes produces audio (and may want to use of
browser-provided facilities, once they become available), I may have a
reasonable interest in interoperability between browsers.

(The Enrichment Center once again reminds you that codec hell is a real
place where your web application will be sent at the first sign of
defiance.)

On further thought, “state of the art lock-in” may not be as bad as you
might fear: First, bandwidth and storage space are becoming cheaper
over time; second, there is something as “good enough”, GZIP / DEFLATE
or MP3 being such examples that serve us for over 15 years each — even
though better specifications (Vorbis, 7z) clearly exist.

Even CPIO is used by my modern desktop system and that was defined
around the mid-80ies (or so Wikipedia tells me, I certainly wasn't
released at that point).


Cheers,
-- 
Nils Dagsson Moskopp // erlehmann

Re: [whatwg] need a way to set output format from StreamRecorder

[Kevin Marks]

On Mon, Feb 14, 2011 at 2:39 PM, Ian Hickson  wrote:

> On Fri, 19 Nov 2010, Per-Erik Brodin wrote:
> >
> > We are about to start implementing stream.record() and StreamRecorder.
> > The spec currently says that “the file must be in a format supported by
> > the user agent for use in audio and video elements” which is a
> > reasonable restriction. However, there is currently no way to set the
> > output format of the resulting File that you get from recorder.stop().
> > It is unlikely that specifying a default format would be sufficient if
> > you in addition to container formats and codecs consider resolution,
> > color depth, frame rate etc. for video and sample size and rate, number
> > of channels etc. for audio.
> >
> > Perhaps an argument should be added to record() that specifies the
> > output format from StreamRecorder as a MIME type with parameters? Since
> > record() should probably throw when an unsupported type is supplied, it
> > would perhaps be useful to have a canRecordType() or similar to be able
> > to test for supported formats.
>
> I haven't added anything here yet, mostly because I've no idea what to
> add. The ideal situation here is that we have one codec that everyone can
> read and write and so don't need anything, but that may be hopelessly
> optimistic.


That isn't the ideal, as it locks us into the current state of the art
forever. The ideal is to enable multiple codecs +formats that can be swapped
out over time. That said, uncompressed audio is readily codifiable, and we
could pick a common file format, sample rate, bitdepth and channel caount
specification.


In the meantime I encourage implementors to experiment with
> this (with all the APIs vendor-prefixed of course) to work out what the
> API should look like. Implementation experience is the main thing that
> will drive this forward, I think.
>
>
That is a fair point.

Re: [whatwg] need a way to set output format from StreamRecorder

[Ian Hickson]

On Fri, 19 Nov 2010, Per-Erik Brodin wrote:
>
> We are about to start implementing stream.record() and StreamRecorder. 
> The spec currently says that “the file must be in a format supported by 
> the user agent for use in audio and video elements” which is a 
> reasonable restriction. However, there is currently no way to set the 
> output format of the resulting File that you get from recorder.stop(). 
> It is unlikely that specifying a default format would be sufficient if 
> you in addition to container formats and codecs consider resolution, 
> color depth, frame rate etc. for video and sample size and rate, number 
> of channels etc. for audio.
> 
> Perhaps an argument should be added to record() that specifies the 
> output format from StreamRecorder as a MIME type with parameters? Since 
> record() should probably throw when an unsupported type is supplied, it 
> would perhaps be useful to have a canRecordType() or similar to be able 
> to test for supported formats.

I haven't added anything here yet, mostly because I've no idea what to 
add. The ideal situation here is that we have one codec that everyone can 
read and write and so don't need anything, but that may be hopelessly 
optimistic. In the meantime I encourage implementors to experiment with 
this (with all the APIs vendor-prefixed of course) to work out what the 
API should look like. Implementation experience is the main thing that 
will drive this forward, I think.

Cheers,
-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] need a way to set output format from StreamRecorder

[Anne van Kesteren]

On Fri, 26 Nov 2010 19:01:40 +0100, Per-Erik Brodin  
 wrote:
A Stream can be treated as an abstract representation of a media stream.  
When a Stream is to be transported over a peer-to-peer connection, the  
format can be negotiated between the peers. In the current  
ConnectionPeer API, such format negotiation would be transparent to the  
API. If we would specify a single resolution for video, for example,  
that resolution may be to high for some mobile devices to encode in  
real-time. A mismatch in supported formats is just one reason why a  
peer-to-peer transport may fail, but that doesn't mean that the peers  
can't communicate. When relaying through a server you can interoperate  
with anything.


Via a server, maybe, if the author took care of having a transcoder there.  
Ideally P2P is without such an intermediary. And although authors can  
exclude parties now by using proprietary plug-ins, I am not convinced we  
should make that an intrinsic part of the web. Not finding a codec /  
simply using VP8 will make that a reality however.



If you are referring to sendFile(file) on ConnectionPeer, the file may  
just as well come from the user's hard drive via  and  
thus it will be up to the application to ensure that whatever is sent to  
the other peer is usable there.


That is quite a different scenario from exchanging a live media feed.


--
Anne van Kesteren
http://annevankesteren.nl/

Re: [whatwg] need a way to set output format from StreamRecorder

[Anne van Kesteren]

On Sat, 27 Nov 2010 21:26:58 +0100, Silvia Pfeiffer  
 wrote:

As for choosing a single codec - if we end up not being able to agree
on a single compressed codec for audio/video streaming, we will get
into a situation where all your involved parties in a video or audio
conference have to use the same browser (or select from a restricted
group of browsers only) for a live communication. I guess it's
workable, but kinda strange for an open platform like the Web.


I would not really even want to consider that as an acceptable outcome, to  
be honest.



--
Anne van Kesteren
http://annevankesteren.nl/

Re: [whatwg] need a way to set output format from StreamRecorder

[Silvia Pfeiffer]

Incidentally, I agree that recording audio should best be kept to an
uncompressed format, because I can see great browser applications for
audio editing.

As for choosing a single codec - if we end up not being able to agree
on a single compressed codec for audio/video streaming, we will get
into a situation where all your involved parties in a video or audio
conference have to use the same browser (or select from a restricted
group of browsers only) for a live communication. I guess it's
workable, but kinda strange for an open platform like the Web.

Cheers,
Silvia.


On Sun, Nov 28, 2010 at 7:20 AM, Silvia Pfeiffer
 wrote:
> In deed for audio all browsers except IE9 currently support WAV in
> their media elements, which makes it a reasonable recording format and
> acceptable for saving to the server. For communication between browser
> instances - in particular when used for conferenceing - I can see the
> need for a compressed format.
>
> Speex is a reasonable low-bandwidth codec, but for speech only and not
> for general audio. I would wait with choosing a low-bandwidth codec
> until the IETF's new "Internet Wideband Audio Codec" [1] Working Group
> finalizes their codec definition, since that will be a low-bandwidth
> codec not restricted to speech. It will be an unencumbered codec
> called "Opus" and is making great progress, see
> http://www.ietf.org/mail-archive/web/codec/current/msg02029.html .
>
> Cheers,
> Silvia.
>
> [1] http://datatracker.ietf.org/wg/codec/charter/
>
> On Sun, Nov 28, 2010 at 6:51 AM, Kevin Marks  wrote:
>> For Audio at least, supporting uncompressed should be possible and
>> uncontroversial, as there are clearly no patent issues here. Anyone serious
>> about recording and processing audio would not consider recording compressed
>> audio nowadays. T
>> There are several widely used raw audio formats (.au, WAV, AIFF, AVI) that
>> can wrap into a filestream, and there are of course the issues of sample
>> rate, channel count and bit resolution, but compared to codec issues these
>> are relatively straightforward from an engineering point of view, and not
>> tied up with licensing issues.
>> Raw video is more of a problem at present, given common bandwidth
>> constraints, but if we are interested in providing for image manipulation
>> APIs, having pixel formats that map to video better than RGBA may be needed.
>> The enumeration at
>> http://developer.apple.com/quicktime/icefloe/dispatch020.html
>> may be helpful here.
>> On Fri, Nov 26, 2010 at 11:10 AM, Nils Dagsson Moskopp
>>  wrote:
>>>
>>> Silvia Pfeiffer  schrieb am Thu, 25 Nov 2010
>>> 20:01:37 +1100:
>>>
>>> > Also, implementing WebM or Ogg Theora encoding is just as royalty-free
>>> > as decoding them, so Mozilla, Opera and Google wouldn't need to worry
>>> > there.
>>>
>>> Slightly offtopic: Anyone considering the low-bandwith audio use case?
>>> Surely, speex might be useful here — even a throttled UMTS connection
>>> suffices for VoIP.
>>>
>>> > So, the browsers would implement support for those codecs for which
>>> > they already implement decoding support - maybe with the exception of
>>> > Chrome which decode MPEG-4, but may not want to encode it, since it
>>> > might mean extra royalties.
>>>
>>> And probably less WebM content, too boot. Decoding, but not encoding
>>> MPEG formats could certainly fit into a royalty-free formats agenda,
>>> depending on the level of aggressiveness Google is wishing to take.
>>>
>>> > It would be nice if we could all, say, encode WebM, but I don't see
>>> > that happening.
>>>
>>> I see what you did there.
>>>
>>>
>>> Greetings,
>>> --
>>> Nils Dagsson Moskopp // erlehmann
>>> 
>>
>>
>

Re: [whatwg] need a way to set output format from StreamRecorder

[Silvia Pfeiffer]

In deed for audio all browsers except IE9 currently support WAV in
their media elements, which makes it a reasonable recording format and
acceptable for saving to the server. For communication between browser
instances - in particular when used for conferenceing - I can see the
need for a compressed format.

Speex is a reasonable low-bandwidth codec, but for speech only and not
for general audio. I would wait with choosing a low-bandwidth codec
until the IETF's new "Internet Wideband Audio Codec" [1] Working Group
finalizes their codec definition, since that will be a low-bandwidth
codec not restricted to speech. It will be an unencumbered codec
called "Opus" and is making great progress, see
http://www.ietf.org/mail-archive/web/codec/current/msg02029.html .

Cheers,
Silvia.

[1] http://datatracker.ietf.org/wg/codec/charter/

On Sun, Nov 28, 2010 at 6:51 AM, Kevin Marks  wrote:
> For Audio at least, supporting uncompressed should be possible and
> uncontroversial, as there are clearly no patent issues here. Anyone serious
> about recording and processing audio would not consider recording compressed
> audio nowadays. T
> There are several widely used raw audio formats (.au, WAV, AIFF, AVI) that
> can wrap into a filestream, and there are of course the issues of sample
> rate, channel count and bit resolution, but compared to codec issues these
> are relatively straightforward from an engineering point of view, and not
> tied up with licensing issues.
> Raw video is more of a problem at present, given common bandwidth
> constraints, but if we are interested in providing for image manipulation
> APIs, having pixel formats that map to video better than RGBA may be needed.
> The enumeration at
> http://developer.apple.com/quicktime/icefloe/dispatch020.html
> may be helpful here.
> On Fri, Nov 26, 2010 at 11:10 AM, Nils Dagsson Moskopp
>  wrote:
>>
>> Silvia Pfeiffer  schrieb am Thu, 25 Nov 2010
>> 20:01:37 +1100:
>>
>> > Also, implementing WebM or Ogg Theora encoding is just as royalty-free
>> > as decoding them, so Mozilla, Opera and Google wouldn't need to worry
>> > there.
>>
>> Slightly offtopic: Anyone considering the low-bandwith audio use case?
>> Surely, speex might be useful here — even a throttled UMTS connection
>> suffices for VoIP.
>>
>> > So, the browsers would implement support for those codecs for which
>> > they already implement decoding support - maybe with the exception of
>> > Chrome which decode MPEG-4, but may not want to encode it, since it
>> > might mean extra royalties.
>>
>> And probably less WebM content, too boot. Decoding, but not encoding
>> MPEG formats could certainly fit into a royalty-free formats agenda,
>> depending on the level of aggressiveness Google is wishing to take.
>>
>> > It would be nice if we could all, say, encode WebM, but I don't see
>> > that happening.
>>
>> I see what you did there.
>>
>>
>> Greetings,
>> --
>> Nils Dagsson Moskopp // erlehmann
>> 
>
>

Re: [whatwg] need a way to set output format from StreamRecorder

[Kevin Marks]

For Audio at least, supporting uncompressed should be possible and
uncontroversial, as there are clearly no patent issues here. Anyone serious
about recording and processing audio would not consider recording compressed
audio nowadays. T

There are several widely used raw audio formats (.au, WAV, AIFF, AVI) that
can wrap into a filestream, and there are of course the issues of sample
rate, channel count and bit resolution, but compared to codec issues these
are relatively straightforward from an engineering point of view, and not
tied up with licensing issues.

Raw video is more of a problem at present, given common bandwidth
constraints, but if we are interested in providing for image manipulation
APIs, having pixel formats that map to video better than RGBA may be needed.
The enumeration at
http://developer.apple.com/quicktime/icefloe/dispatch020.html
may be helpful here.

On Fri, Nov 26, 2010 at 11:10 AM, Nils Dagsson Moskopp <
nils-dagsson-mosk...@dieweltistgarnichtso.net> wrote:

> Silvia Pfeiffer  schrieb am Thu, 25 Nov 2010
> 20:01:37 +1100:
>
> > Also, implementing WebM or Ogg Theora encoding is just as royalty-free
> > as decoding them, so Mozilla, Opera and Google wouldn't need to worry
> > there.
>
> Slightly offtopic: Anyone considering the low-bandwith audio use case?
> Surely, speex might be useful here — even a throttled UMTS connection
> suffices for VoIP.
>
> > So, the browsers would implement support for those codecs for which
> > they already implement decoding support - maybe with the exception of
> > Chrome which decode MPEG-4, but may not want to encode it, since it
> > might mean extra royalties.
>
> And probably less WebM content, too boot. Decoding, but not encoding
> MPEG formats could certainly fit into a royalty-free formats agenda,
> depending on the level of aggressiveness Google is wishing to take.
>
> > It would be nice if we could all, say, encode WebM, but I don't see
> > that happening.
>
> I see what you did there.
>
>
> Greetings,
> --
> Nils Dagsson Moskopp // erlehmann
> 
>

Re: [whatwg] need a way to set output format from StreamRecorder

[Nils Dagsson Moskopp]

Silvia Pfeiffer  schrieb am Thu, 25 Nov 2010
20:01:37 +1100:

> Also, implementing WebM or Ogg Theora encoding is just as royalty-free
> as decoding them, so Mozilla, Opera and Google wouldn't need to worry
> there.

Slightly offtopic: Anyone considering the low-bandwith audio use case?
Surely, speex might be useful here — even a throttled UMTS connection
suffices for VoIP.

> So, the browsers would implement support for those codecs for which
> they already implement decoding support - maybe with the exception of
> Chrome which decode MPEG-4, but may not want to encode it, since it
> might mean extra royalties.

And probably less WebM content, too boot. Decoding, but not encoding
MPEG formats could certainly fit into a royalty-free formats agenda,
depending on the level of aggressiveness Google is wishing to take.

> It would be nice if we could all, say, encode WebM, but I don't see
> that happening.

I see what you did there.


Greetings,
-- 
Nils Dagsson Moskopp // erlehmann



signature.asc
Description: PGP signature

Re: [whatwg] need a way to set output format from StreamRecorder

[Per-Erik Brodin]

On 2010-11-23 18:24, Anne van Kesteren wrote:

On Fri, 19 Nov 2010 19:50:42 +0100, Per-Erik Brodin
  wrote:

We are about to start implementing stream.record() and StreamRecorder.
The spec currently says that “the file must be in a format supported by
the user agent for use in audio and video elements” which is a
reasonable restriction. However, there is currently no way to set the
output format of the resulting File that you get from recorder.stop().
It is unlikely that specifying a default format would be sufficient if
you in addition to container formats and codecs consider resolution,
color depth, frame rate etc. for video and sample size and rate, number
of channels etc. for audio.

Perhaps an argument should be added to record() that specifies the
output format from StreamRecorder as a MIME type with parameters? Since
record() should probably throw when an unsupported type is supplied, it
would perhaps be useful to have a canRecordType() or similar to be able
to test for supported formats.


But if we want interoperability for streams, also taking into account P2P
messaging, we need a single format. Otherwise users with different
browsers could end up not being able to communicate.


A Stream can be treated as an abstract representation of a media stream. 
When a Stream is to be transported over a peer-to-peer connection, the 
format can be negotiated between the peers. In the current 
ConnectionPeer API, such format negotiation would be transparent to the 
API. If we would specify a single resolution for video, for example, 
that resolution may be to high for some mobile devices to encode in 
real-time. A mismatch in supported formats is just one reason why a 
peer-to-peer transport may fail, but that doesn't mean that the peers 
can't communicate. When relaying through a server you can interoperate 
with anything.


If you are referring to sendFile(file) on ConnectionPeer, the file may 
just as well come from the user's hard drive via  and 
thus it will be up to the application to ensure that whatever is sent to 
the other peer is usable there.


//Per-Erik

Re: [whatwg] need a way to set output format from StreamRecorder

[Silvia Pfeiffer]

On Thu, Nov 25, 2010 at 6:37 PM, Nils Dagsson Moskopp
 wrote:
> Silvia Pfeiffer  schrieb am Thu, 25 Nov 2010
> 14:05:18 +1100:
>
>> Can the decision for a file format be taken completely separately from
>> the codec decision for the  or  element, I wonder?
>
> I believe the royalties for encoders are usually higher than the
> royalties for decoders (where royalties apply, that is: with
> proprietary standards).
>
> Also, I doubt that any self-respecting entities opposed to the
> implementation of decoders for certain royalty-free A/V formats would
> include the corresponding encoders — the risk being that more content
> would be created in formats that their own browsers could not render.

Naturally a browser that doesn't decode MPEG-4 would not implement
MPEG-4 encoding.

Apple probably already pay the max royalties for MPEG-4, so they
probably won't worry about MPEG-4 encoding support in Safari.

Also, implementing WebM or Ogg Theora encoding is just as royalty-free
as decoding them, so Mozilla, Opera and Google wouldn't need to worry
there.

So, the browsers would implement support for those codecs for which
they already implement decoding support - maybe with the exception of
Chrome which decode MPEG-4, but may not want to encode it, since it
might mean extra royalties.

It would be nice if we could all, say, encode WebM, but I don't see
that happening.

Cheers,
Silvia.

Re: [whatwg] need a way to set output format from StreamRecorder

[Nils Dagsson Moskopp]

Silvia Pfeiffer  schrieb am Thu, 25 Nov 2010
14:05:18 +1100:

> Can the decision for a file format be taken completely separately from
> the codec decision for the  or  element, I wonder?

I believe the royalties for encoders are usually higher than the
royalties for decoders (where royalties apply, that is: with
proprietary standards).

Also, I doubt that any self-respecting entities opposed to the
implementation of decoders for certain royalty-free A/V formats would
include the corresponding encoders — the risk being that more content
would be created in formats that their own browsers could not render.

> My assumption is that a StreamRecorder will get video (or audio) from
> a device and it will be possible to hand it on to a  (or
> ) element for display. Since these elements supports specific
> codecs, wouldn't that imply that the StreamRecorder needs to save it
> in those formats, too? Unless, of course, we want to introduce some
> kind of transcoding element that is plugged between a recorded stream
> and a  () element.

I see your decodebin and raise you a fully self-aware media framework ;)


Cheers,
-- 
Nils Dagsson Moskopp // erlehmann



signature.asc
Description: PGP signature

Re: [whatwg] need a way to set output format from StreamRecorder

[Silvia Pfeiffer]

On Wed, Nov 24, 2010 at 4:24 AM, Anne van Kesteren  wrote:
> On Fri, 19 Nov 2010 19:50:42 +0100, Per-Erik Brodin
>  wrote:
>>
>> We are about to start implementing stream.record() and StreamRecorder. The
>> spec currently says that “the file must be in a format supported by the user
>> agent for use in audio and video elements” which is a reasonable
>> restriction. However, there is currently no way to set the output format of
>> the resulting File that you get from recorder.stop(). It is unlikely that
>> specifying a default format would be sufficient if you in addition to
>> container formats and codecs consider resolution, color depth, frame rate
>> etc. for video and sample size and rate, number of channels etc. for audio.
>>
>> Perhaps an argument should be added to record() that specifies the output
>> format from StreamRecorder as a MIME type with parameters? Since record()
>> should probably throw when an unsupported type is supplied, it would perhaps
>> be useful to have a canRecordType() or similar to be able to test for
>> supported formats.
>
> But if we want interoperability for streams, also taking into account P2P
> messaging, we need a single format. Otherwise users with different browsers
> could end up not being able to communicate.

Can the decision for a file format be taken completely separately from
the codec decision for the  or  element, I wonder?

My assumption is that a StreamRecorder will get video (or audio) from
a device and it will be possible to hand it on to a  (or
) element for display. Since these elements supports specific
codecs, wouldn't that imply that the StreamRecorder needs to save it
in those formats, too? Unless, of course, we want to introduce some
kind of transcoding element that is plugged between a recorded stream
and a  () element.

Cheers,
Silvia.

Re: [whatwg] need a way to set output format from StreamRecorder

[Anne van Kesteren]

On Fri, 19 Nov 2010 19:50:42 +0100, Per-Erik Brodin  
 wrote:
We are about to start implementing stream.record() and StreamRecorder.  
The spec currently says that “the file must be in a format supported by  
the user agent for use in audio and video elements” which is a  
reasonable restriction. However, there is currently no way to set the  
output format of the resulting File that you get from recorder.stop().  
It is unlikely that specifying a default format would be sufficient if  
you in addition to container formats and codecs consider resolution,  
color depth, frame rate etc. for video and sample size and rate, number  
of channels etc. for audio.


Perhaps an argument should be added to record() that specifies the  
output format from StreamRecorder as a MIME type with parameters? Since  
record() should probably throw when an unsupported type is supplied, it  
would perhaps be useful to have a canRecordType() or similar to be able  
to test for supported formats.


But if we want interoperability for streams, also taking into account P2P  
messaging, we need a single format. Otherwise users with different  
browsers could end up not being able to communicate.



--
Anne van Kesteren
http://annevankesteren.nl/