Re: [whatwg] Add naturalOrientation property to img
Maybe someone should ask the browser vendors how many img formats they support and what the code footprint memory overhead would be for adding rotation support for those which are likely to need it at whatever confidence level you feel is appropriate. On Fri, Aug 26, 2011 at 9:57 AM, Glenn Maynard gl...@zewt.org wrote: On Fri, Aug 26, 2011 at 11:37 AM, Tab Atkins Jr. jackalm...@gmail.comwrote: Summary === The img element should expose a readonly naturalOrientation property that returns the EXIF orientation of the image (an integer 1-8), or 1 if the image has no EXIF orientation metadata or it is corrupted or the image type doesn't expose EXIF data at all. Rather than baking magic EXIF constants into the API, would it make more sense to translate to degrees? -- Glenn Maynard
Re: [whatwg] Recording interface (Re: Peer-to-peer communication, video conferencing, and related topics (2))
Sorry for the top posting, but I would like to reiterate my considered
opinion that Speex be supported for recording. It is the standard
format available from Adobe Flash recording, low bandwidth, open
source and unencumbered, efficient, and it is high quality for its
bandwidth.
On Tue, Mar 29, 2011 at 11:37 PM, Stefan Håkansson LK
stefan.lk.hakans...@ericsson.com wrote:
-Original Message-
From: whatwg-boun...@lists.whatwg.org
[mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of
whatwg-requ...@lists.whatwg.org
Sent: den 29 mars 2011 20:33
To: whatwg@lists.whatwg.org
Subject: whatwg Digest, Vol 84, Issue 69
I also believe that the recording interface should be
removed from
this part of the specification; there should be no requirement
that all streams be recordable.
Recording of streams is needed for some use cases unrelated
to video
conferencing, such as recording messages.
Having a recording function is needed in multiple use cases;
I think we all agree on that.
This is mostly a matter of style, which I'm happy to defer on.
The streams should be regarded as a control surface,
not as a data
channel; in many cases, the question of what is the
format of the stream at this point
is literally unanswerable; it may be represented as hardware
states, memory buffers, byte streams, or something
completely different.
Agreed.
Recording any of these requires much more specification
than just
record here.
Could you elaborate on what else needs specifying?
One thing I remember from an API design talk I viewed:
An ability to record to a file means that the file format is
part of your API.
For instance, for audio recording, it's likely that you want
control over whether the resulting file is in Ogg Vorbis
format or in MP3 format; for video, it's likely that you may
want to specify that it will be stored using the VP8 video
codec, the Vorbis audio codec and the Matroska container
format. These desires have to be communicated to the
underlying audio/video engine, so that the proper transforms
can be inserted into the processing stream, and I think they
have to be communicated across this interface; since the
output of these operations is a blob without any inherent
type information, the caller has to already know which format
the media is in.
This is absolutely correct, and it is not only about codecs or
container formats. Maybe you need to supply info like audio
sampling rate, video frame rate, video resolution, ...
There was an input on this already last November:
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-November/029069.html
Clearer?
--
Message: 2
Date: Tue, 29 Mar 2011 15:27:58 +0200
From: Wilhelm Joys Andersen wilhel...@opera.com
To: whatwg@lists.whatwg.org
Subject: [whatwg] details, summary and styling
Message-ID: op.vs3w0wvrm3w...@kunnskapet.oslo.osa
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes
Hi,
I'm currently writing tests in preparation for Opera's implementation
of details and summary. In relation to this, I have a few
questions
about issues that, as far as I can tell, are currently
undefined in the
specification.
The spec says:
If there is no child summary element [of the details element], the
user agent should provide its own legend (e.g. Details). [1]
How exactly should this legend be provided? Should the user agent add
an implied summary element to the DOM, similar to tbody, a
pseudo-element, or a magic non-element behaving differently from both
of the above? In the current WebKit implementation[2], the UA-provided
legend behaves inconsistently from from an author-provided summary
in the following ways:
* Although it can be styled with rules applying to
summary, it does
not respond to :hover or :first-child.
* With regards to text selection, it behaves more like an input
type='submit' than a user-provided summary. Text within this
implied element may only be selected _together_ with the text
preceding and following it.
* A different mouse cursor is used.
This indicates that it is slightly more magic than I would prefer. I
believe a closer resemblance to an ordinary element would be more
convenient for authors - a ::summary pseudo element with Details as
its content() might be the cleanest approach, although that would
require a few more bytes in the author's stylesheet to cater to both
author- and UA-defined summaries:
summary, ::summary {
color: green;
}
Furthermore, the rendering spec says:
The first container is expected to contain at least one line box,
and that line box is expected to contain a disclosure
widget (typically
a triangle), horizontally positioned within the left padding of the
details element. [3]
For user agents aiming to support the suggested default rendering, how
should the disclosure widget be embedded?
Re: [whatwg] default audio upload format (was Fwd: The Media Capture API Working Draft)
Most of the MIME types that support multiple channels and sample rates have registered parameters for selecting those. Using a PCM format such as audio/L16 (CD/Red Book audio) as a default would waste a huge amount of network bandwidth, which translates directly into money for some users. On Fri, Sep 3, 2010 at 2:19 PM, David Singer sin...@apple.com wrote: I agree that if the server says it accepts something, then it should cover at least the obvious bases, and transcoding at the server side is not very hard. However, I do think tht there needs to be some way to protect the server (and user, in fact) from mistakes etc. If the server was hoping for up to 10 seconds of 8kHz mono voice to use as a security voice-print, and the UA doesn't cut off at 10 seconds, records at 48 Khz stereo, and the user forgets to hit 'stop', quite a few systems might be surprised (and maybe charge for) the size of the resulting file. It's also a pain at the server to have to sample-rate convert, downsample to mono, and so on, if the terminal could do it. On Sep 3, 2010, at 14:08 , Roger Hågensen wrote: On 2010-09-01 21:34, David Singer wrote: seems like a comma-separated list is the right way to go, and that audio/* should mean what it says -- any kind of audio (whether that is useful or not remains to be seen). I would suggest that this is likely to be used for short captures, and that uncompressed (such as a WAV file or AVI with PCM or u-law audio) should be the recommended format. If your usage is for longer captures or more specific situations, then indicate a suitable codec. Shouldn't there be statements about channels (mono, stereo, more), sampling rate (8 kHz speech, 16 kHz wideband speech, 44.1 CD-quality, 96 kHz bat-quality) and so on? Hmm! Channels, bits, frequency should be optional in my opinion, (and with a recommendation for a default, stereo 16bit 44.1KHz which is the legacy standard for audio in most formats I guess, or maybe 48KHz as most soundcards seem to be these days?) In most cases a service will either A. use it as it's received (since most computer systems can play back pretty much anything), or B. it's transcoded/converted into one or more formats by the service. (like Youtube does etc.) In other words I am assuming that if the server accept for example the WAV format then it actually fully support the WAV format (at least the PCM audio part). Ditto with MP3, AAC, Ogg, FLAC, Speex etc. So any quality, channels, bits, frequency specified in the accept would just be what the server prefers (suggested default, or for best quality/best effort scenario), but the full format should be supported and accepted if asked for. Now whether the service takes advantage of surround rear recording is up to the service, if it simply discards that, takes the front channels and mix them to mono then that is up to the service and the user to decide/negotiate about rather than the browser. -- Roger Rescator Hågensen. Freelancer - http://EmSai.net/ David Singer Multimedia and Software Standards, Apple Inc.
Re: [whatwg] default audio upload format (was Fwd: The Media Capture API Working Draft)
On Tue, Aug 31, 2010 at 11:24 PM, Roger Hågensen resca...@emsai.net wrote: On 2010-08-31 22:11, James Salsman wrote: Does anyone object to form input type=file accept=audio/*;capture=microphone using Speex as a default, as if it were specified accept=audio/x-speex;quality=7;bitrate=16000;capture=microphone or to allowing the requesting of different speex qualities and bitrates using those mime type parameters? Speex at quality=7 is a reasonable open source default audio vocoder. Runner-up alternatives would include audio/ogg, which is a higher bandwidth format appropriate for multiple speakers or polyphonic music; audio/mpeg, a popular but encumbered format; audio/wav, a union of several different sound file formats, some of which are encumbered; etc. Actually, wouldn't accept=audio/*;capture=microphone basically indicate that the server wish to accept anything as audio? Yes; that doesn't encourage implementors to implement. However, as it was agreed on by two different browser developer representatives, it's the best way forward. The proper way however would be to do: accept=audio/flac, audio/wav, audio/ogg, audio/aac, audio/mp3;capture=microphone indicating all the audio formats the server can handle. I agree that the specifications should allow a comma- or space-separated list of MIME types. I have no idea why that was specifically disallowed in the most recent HTML5 specification, and think that decision should be reversed before publication. I also think capture would work a lot better as an attribute of the input type=file element than a MIME type parameter. Although I guess that audio/* could be taken as a sign that the browser should negotiate directly with the server about the preferred format to use. (Is POST HEADER request supported even?) Not for multipart/form-encoded input type=file uploads, sadly. There's a way to do that specified in http://w3.org/TR/device-upload with the alternates form POST parameter which the browser would send back to the server when the requested type(s) were unavailable. I guess that is a content negotiation feature, but it seemed unlikely that people would need it for that because ideally the server would specify all the types it could accept, as you pointed out. Best regards, James Salsman
[whatwg] default audio upload format (was Fwd: The Media Capture API Working Draft)
Does anyone object to form input type=file accept=audio/*;capture=microphone using Speex as a default, as if it were specified accept=audio/x-speex;quality=7;bitrate=16000;capture=microphone or to allowing the requesting of different speex qualities and bitrates using those mime type parameters? Speex at quality=7 is a reasonable open source default audio vocoder. Runner-up alternatives would include audio/ogg, which is a higher bandwidth format appropriate for multiple speakers or polyphonic music; audio/mpeg, a popular but encumbered format; audio/wav, a union of several different sound file formats, some of which are encumbered; etc. -- Forwarded message -- From: James Salsman jsals...@gmail.com Date: Tue, Aug 31, 2010 at 10:08 AM Subject: Re: The Media Capture API Working Draft To: Ilkka Oksanen ilkka.oksa...@nokia.com, Frederick Hirsch frederick.hir...@nokia.com, Robin Berjon ro...@berjon.com Cc: public-device-a...@w3.org public-device-a...@w3.org, Ian Hickson i...@hixie.ch, Tran, Dzung D dzung.d.t...@intel.com On Tue, Aug 31, 2010 at 2:49 AM, Ilkka Oksanen ilkka.oksa...@nokia.com wrote: 31/08/2010 03:43, ext James Salsman kirjoitti: On Mon, Aug 30, 2010 at 6:58 AM, Ilkka Oksanen ilkka.oksa...@nokia.com wrote: After the Media Capture split the HTML Media Capture spec has been released as a Working Draft. I would like now to propose the same for the second specification. That is the Media Capture API http://dev.w3.org/2009/dap/camera/Overview-API.html please let us know if there any changes that you would see implemented before the publification. What is the default audio capture format? Speex? If so, at what bit rate and quality level, and how can those be adjusted? Can you please go ahead and add the format [attribute] with a specific default such as format=audio/x-speex;quality=7;bitrate=16000 so that implementors can have some guidance here? Thank you. There is no defined default value for capture format and I'm hesitant to propose such thing. In a similar situation HTML WG and WHATWG were not able to agree on the default video codec. That would be the likely outcome also in here I think. My understanding is that Speex default value has already been approved in changes to the System Information draft which were promised by today from the new editor of that draft. If I'm mistaken about that, please let me know. If there is any contention on this question, I would ask that the chairs resolve it by polling
Re: [whatwg] video application/octet-stream
I would like to speak up for sniffing because I believe in the robustness principle. I don't know why people who are capable of coding hundreds of lines of dense, uncommented javascript can't get their web servers to declare the correct type of their media files, but it happens so often that I think sniffing is completely reasonable. There aren't too many media types without distinctive headers these days, so the actual arguments against sniffing are pretty weak, aren't they? On Tue, Jul 20, 2010 at 3:28 AM, Philip Jägenstedt phil...@opera.com wrote: On Tue, 20 Jul 2010 12:00:41 +0200, Chris Double chris.dou...@double.co.nz wrote: On Tue, Jul 20, 2010 at 9:48 PM, Philip Jägenstedt phil...@opera.com wrote: I'd like to hear from Mozilla, Google and Apple which of these (or other) solutions they would find acceptable. You'll probably get different responses depending on who in Mozilla responds. For example, I prefer option (1) and am against content sniffing. Other's at Mozilla disagree I'm sure. I'd like to hear from those that will actually make the decision and write the code. I could go with (1) if Chrome does too. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML Media Capture prepared for publication
At least one of the browser authors who were saying they were implementing buffered device input and upload from cameras and microphones with input type=file had indicated they might use the accept= parameter, but I can certainly see the benefit of being able to specify the choice of a file from either the filesystem or a device, and a MIME type parameter is as good a place as any to do that. Please forgive the cross posting and top posting, but I also need to ask: Has anyone considered how to restrict onchanged=form.submit() in such forms? I think that is inappropriate for most multipart/form-encoded POSTs, but I don't want to suggest anything that would break existing applications. Furthermore I hope the security considerations sections will soon include a summary of the language we've discussed about prompting for device access with the option for remembered choices for permission grants, along with a way to override and respond to such prompts with device buttons like a shutter key and/or a record key. I understand that the user always has an option to reject buffered and filesystem-based device input by not submitting the form, but it would be spectacular for the web application to have a way to know when the user decided it wouldn't grant access to the device (i.e., if there is background noise or any of dozens of other reasons.) Finally, I strongly prefer granted permission review and revocability and I hope that makes it in to the security considerations section, too. Sincerely, James Salsman On Sat, Jul 17, 2010 at 11:47 AM, ingmar.kli...@telekom.de wrote: Hi Dom, I think we also briefly tackled the issue of making the FormatData attribute of the MediaFile interface read-only (in File all attributes are read-only [1]). Was there a decision against making it read-only? - Ingmar. [1] http://www.w3.org/TR/2009/WD-FileAPI-20091117/#file -Original Message- From: public-device-apis-requ...@w3.org [mailto:public-device-apis-requ...@w3.org] On Behalf Of Dominique Hazael-Massieux Sent: Friday, July 16, 2010 12:15 PM To: public-device-a...@w3.org Subject: HTML Media Capture prepared for publication Hi, The Working Group agreed on targeting to publish HTML Media Capture (which ties input type=file accept=image/*;capture=camera to the camera and the MediaFile interface) next week as an updated draft of http://www.w3.org/TR/capture-api/ . I've finished preparing the draft with the changes agreed upon during the discussions in the F2F this morning at: http://dev.w3.org/2009/dap/camera/ This includes: - Changed title to HTML Media Capture - Change capture attribute to capture parameter of accept attribute with values in camera, microphone, camcorder - Remove type attribute from FormatData (redundant with Blob.type) - added note on relation to HTML5 (MediaFile interface, capture parameter) - added note on possible relationship to Media Ontology API - Removed the trusted environments example Please let me know if there any last minute changes that you think would be useful to bring to the document before a new public draft. Dom
Re: [whatwg] HTML Device element status
On Tue, Jul 6, 2010 at 5:18 AM, Anne van Kesteren ann...@opera.com wrote: There are some advantages with input, but overall the design is ugly. input type=file is buffered, which would seem to exclude the possibility of onchange=form.submit() in any of its forms' elements, but is otherwise parsimonious, while device is its unbuffered counterpart. Why is one method of transmission any more or less ugly than the other? Buffering can substantially reduce bandwidth. Is there any reason not to protect both them with the same privacy and security authorization dialogs, and render them mostly the same, except for audio/* and video/* input you might have record/pause/play/submit while device would have record/pause? For image/* the differences are less clear to me: perhaps input would have a viewfinder, (expandable on mobiles) shutter button, a filesystem browser button, and an (optional?) submit button, but an image/* device might only have a viewfinder and a shutter button. For the case of a camera, it would seem to me that the buffered approach is also superior, but the unbuffered device approach would allow audio and video teleconferencing. Also, someone said it would be a good idea to mute audio input and output from any hidden tab. I think this may be a reasonable user option (people who listen to podcasts might not like it), and wonder if microphone or other audio input should be muted from any window and tab without top-most focus and exposure. Does anyone have thoughts on that question?
Re: [whatwg] input type=location proposals
On Thu, Jun 24, 2010 at 1:45 PM, Jeremy Keith jer...@adactio.com wrote: Michelango wrote: ... 3. Maps data are often non-free and non-open, reliable maps data are always non-free and non-open. The second clause of point 3 is demonstrably false. Said demonstration is http://www.openstreetmap.org/ which in many cases (e.g. the town I live in) has better, more up-to-date reliable data than its non-free, non-open counterparts. See also: Wikipedia, blogs, and much of the World Wide Web The Wikimedia California Chapter initial funding proposal has some interactive map work in it, if people are interested. I've been trying to raise money for it but it's been going pretty slow. Anyone who cares can find it on line. It needs $150,000 plus management salary to get going as a viable concern. The point here is that even free things take time and/or money to get right and and get open right. Locations should have security considerations similar to contacts. It seems reasonable to assume that people are going to want to download them more than upload them, unless they have a very tightly controlled assumption about of the list of recipients, and some assurance that their assumptions are correct. Also locations share a lot in common with the user's contacts, of course, including the fact that at least one is associated with the user. (Telepresence is an interesting example of a situation where a person may be associated with multiple locations, but those are so rare that example seems contrived.) Regards, James Salsman
Re: [whatwg] Do we really need a device element?
On Wed, Jun 16, 2010 at 10:43 AM, Bjartur Thorlacius svartma...@gmail.com wrote: On 6/14/10, James Salsman jsals...@gmail.com wrote: ... I [had been earlier] persuaded that the device element is unnecessary, given recent announcements for the input type=file accept=...;source=... type specification proposed by Android and Firefox developers. Does anyone have any reasons to the contrary? A device element with a type parameter would be useful when the HTML author is unaware of the target browsers' choice of specific elements such as embed, object, video, microphone, camera, camcorder etc. to control placement of, for example, audio volume and recording level controls, microphone visual feedback, video display viewports, camera and camcorder viewfinders, etc. for real-time device access, which does seem to be very reasonable to do in HTML instead of Flash. Use cases include teleconferencing, screen grabs (device type=image;source=desktop?), maybe shared whiteboards (device type=pointer;source=canvas-id producing coordinate streams and up/down codes?) Real-time camcorder upload has as compelling use case as buffered form-based input type=file accept=video;source=webcam does under lower bandwidth situations. People will want both, so I am not ready to write off the device element yet. Are file inputs defined to be more buffered than devices? Yes. Where? At this point, it's either DAP or WebApps, and I am trying to figure out which. The input type=file element has been buffered, asynchronous (per POST, not per javascript callback), multipart/form-encoded since HTML 3, with the accept=... MIME type list attribute appearing in HTML 3.2. IMO a streaming capability should rather be added to form than adding a brand-new device element. Streaming forms? There is input onChange=Form.submit() ..., but that is not really appropriate for file uploads and I haven't seen any proposals along those lines. (Someone please correct me if there are any!) To the extent that we are supposed to obtain interoperability before promoting standards, we need to follow current practice, which is closer to the sockets API as far as emulating the behavior of Adobe Flash goes (if it were up to me, the sockets and the file interface would be merged into a backward-compatible, coherent, and less namespace-cluttering whole, but I think that ship has probably sailed.) The only thing the device element does is hinting that the input should come from alternative sources and provide a new scripting interface. It's also supposed to cause a permissions request (which may typically be a non-blocking (modeless), one-at-a-time prompt for things like microphones or contacts, or a shutter button press for things like cameras. What if you want to use the new scripting interface but not hint that the input should be from alternative sources? Presumably there will be reasonable defaults, or a selection dialog if there is more than one possibility and a default has not been established. The device element represents a device selector, to allow the user to give the page access to a device, for example a video camera. So, it's an UI-widget an UA can show to human users to authorize script access to a device. Where's the definition of a device (couldn't find it in the draft, may be my ignorance). If this becomes a standard, shouldn't there also be a standard for an element for pop-up authorizing UI-widgets? Yes, the same interface is being contemplated for file system, contacts database, calendar item, and photo gallery access at present. If privacy is the reason for this element, as the draft says, why is/would it not be enough to allow requests for devices to fail and allow users not to upload (live) recordings of themselves? I don't understand that question, sorry.
Re: [whatwg] Do we really need a device element?
On Sun, Jun 13, 2010 at 6:36 PM, James Salsman jsals...@gmail.com wrote: ... I [was] persuaded that the device element is unnecessary, given recent announcements for the accept=...;source=... type specification proposed by Android and Firefox developers. Does anyone have any reasons to the contrary? A device element with a type parameter would be useful when the HTML author is unaware of the target browsers' choice of specific elements such as embed, object, video, microphone, camera, camcorder etc. to control placement of, for example, audio volume and recording level controls, microphone visual feedback, video display viewports, camera and camcorder viewfinders, etc. for real-time device access, which does seem to be very reasonable to do in HTML instead of Flash. Use cases include teleconferencing, screen grabs (device type=image;source=desktop?), maybe shared whiteboards (device type=pointer;source=canvas-id producing coordinate streams and up/down codes?) Real-time camcorder upload has as compelling use case as buffered form-based input type=file accept=video;source=webcam does under lower bandwidth situations. People will want both, so I am not ready to write off the device element yet.
Re: [whatwg] Do we really need a device element?
Summary: I have been persuaded that the device element is unnecessary, given recent announcements for the accept=...;source=... type specification proposed by Android and Firefox developers. Does anyone have any reasons to the contrary? On Sun, Jun 13, 2010 at 7:33 AM, Bjartur Thorlacius svartma...@gmail.com wrote: On 6/12/10, James Salsman jsals...@gmail.com wrote: Bjartur, real-time streaming audio uploads enable pronunciation assessment for language instruction during multiplayer online roleplaying games such as those by http://www.8dworld.com Yes, applications may need audio/visual input, hence W3C's WG DAP has created an API for accessing audiovisual devices. Created is as yet too past tense for something that's supposed to be documenting practice, of which the only so far is to use Adobe Flash instead of HTML. As there's already a standard for accessing such devices, what value does device add? Per http://dev.w3.org/html5/html-device/ the device element is a way for a page to request permission for its scripts to access a device. Note that it, as a proposal, isn't finalized. I can certainly see the argument for implicit behavior when either input type=... accept=... form elements or real-time javascript streaming interfaces attempt to access a particular device: The same permission request should occur in either case, so why have a separate element for it? And why should this even be a HTML element in the first place, rather than an (JS) API? I don't have a good answer to that. What do others think? input type=file accept=audio/x-speex;quality=(0-10);bitrate=... is more important, and accept=audio/ogg video/x-webm, video/ogg are all fairly important, too. One non-native English writer to another: I beg your pardon? I mean that Speex, Ogg Vorbis, Google VP8/webm, and possibly Ogg Theora if the W3C patent examination staff finds anything wrong with VP8, should be included in form-based device input and upload, ideally, as well as the real-time streaming javascript interface which seems more ephemeral at this point. On the other hand, audio/wav should not be (it is currently referenced in at least one DAP draft document), because it's a minefield union type, and its subtypes are either too high-bandwidth (such as PCM like audio/L16), too low quality (like a-law or mu-law), have such poor compression artifacts, or are of too poor or uncertain encumbrance status to use. I should have written input type=file accept='audio/x-speex;quality=(0-10);bitrate=...;source=microphone' to reflect the newly-announced Android/Firefox placement of the source= parameter. I would like to see something like that for device type=audio per http://dev.w3.org/html5/html-device/#stream-api (This will be pinned down to a specific codec.) But again, I think the issues raised here are good ones and if anyone knows why there is an actual need for a separate device element, please tell me.
Re: [whatwg] Do we really need a device element?
Bjartur, real-time streaming audio uploads enable pronunciation assessment for language instruction during multiplayer online roleplaying games such as those by http://www.8dworld.com input type=file accept=audio/x-speex;quality=(0-10);bitrate=... is more important, and accept=audio/ogg video/x-webm, video/ogg are all fairly important, too. On Jun 12, 2010 11:15 AM, Bjartur Thorlacius svartma...@gmail.com wrote: What is the use case for device? I´ve searched the archives, read a thread, Googled around and read a document on the subject, but still I haven´t found a use case for it other than it is going to magically increase user privacy. input type=file accept=audio/* is a standard and compatible way to request audio stream(s). I can´t see any drawbacks to using input for audiovisual streams at least. -- kv, - Bjartur
Re: [whatwg] input type=upload (not just files) proposal
On Wed, Jun 9, 2010 at 5:22 AM, Rob Evans r...@mtn-i.com wrote: I think an interesting addition to the input type=upload would be a script-accessible progress value. Either as a percentage, or probably more useful two values, one being the size of the upload and the other being current bytes sent. If you are uploading by reference to URL (which presumably the server would fetch to save bandwidth) then progress information is probably not something that can easily be exposed by the HTML interface. You might want to use an out-of-band ajax-style channel with javascript to show progress instead. Also whilst we're on the subject, how about allowing multiple files to be selected from a single input element? That's been in there since HTML 3.2, I believe, and was implemented by some browsers (IE 5-8?). User interface concerns almost always end up preferring dynamic form behaviors, with variable numbers of input type=file elements added and removed on an as-needed basis before the form is submitted.
Re: [whatwg] What is not possible with HTML5
On Jun 7, 2010 2:34 AM, narendra sisodiya naren...@narendrasisodiya.com wrote: May someone explore what is not possible with HTML5 in spec and in Implementations input type=file accept=audio per HTML 3.2, 4, 5 and http://www.w3.org/TR/device-upload has been around for more than a decade without browser implementation, even though hundreds language instructors and Tim Berners-Lee himself have asked for implementation because it would enable many different kinds of apps. My W3C invited-expert position on the issue is detailed at http://talknicer.com/convAppsPosn.pdf There is a Mozilla bug on this -- https://bugzilla.mozilla.org/show_bug.cgi?id=46135 -- which has been open for a whole decade, with 37 votes, and even had a patch back when it was first opened. Does anyone know how I can update my email address in Mozilla's bugzilla? I am no longer at bovik.org. --James Salsman
Re: [whatwg] WebSockets: UDP
On Fri, Jun 4, 2010 at 4:19 AM, James May wha...@fowlsmurf.net wrote: Couldn't SCTP/DCCP (or a variant) over UDP (for NAT compatibility) work? No, DCCP is much newer than most NAT hardware in operation. When a client user agent is sending UDP, client-initiated TCP streams such as HTTP or HTTPS persistent connections for all data from the server including packet acknowledgment is likely to be the most reliable. Bandwidth from the server to the client is usually greater than or equal to bandwidth from client to server.
Re: [whatwg] WebSockets: UDP
I agree UDP sockets are a legitimate, useful option, with applications far beyond games. In most cases TCP is fine, but adaptive bit-rate vocoders, for example, can use packet loss as an adaptation parameter, and chose only to retransmit some of the more essential packets in cases of congestion. I am not suggesting that javascript applications should implement adaptive bit-rate vocoding (until a fast cross-platform javascript signal processing library is developed, if then) but there are reasons that a web application might want to send both reliable and unreliable traffic; most all of them having to do with adapting to bandwith constraints. On Tue, Jun 1, 2010 at 8:52 AM, John Tamplin j...@google.com wrote: ... How would proxies be handled? UDP is supposed to never be buffered, not even by proxies. Proxies are supposed to simply forward UDP without logging. Lots of them don't forward any UDP, and alot of them probably log the traffic. Even if specs were written and implementations available, how many years would it be before corporate proxies/firewalls supported WebSocket over UDP? Maybe UDP adoption would follow adoption of SIP and RTP. Has anyone measured the current rate of UDP transmission availability from the typical corporate client host? On Tue, Jun 1, 2010 at 1:02 PM, Erik Möller emol...@opera.com wrote: what would the minimal set of limitations be to make a UDP WebSocket browser-safe? -No listen sockets For outgoing-from-client UDP, client-initiated TCP streams for incoming responses and packet acknowledgment may be maximally NAT-safe. -No multicast People will eventually ask for it, but forwarding to it through servers known to be free from NATs is preferable. -Reliable handshake with origin info Nothing about UDP is reliable, you just send packets and hope they get there. -Automatic keep-alives You mean on the incoming-to-client TCP channel in the opposite direction from the UDP traffic? -Reliable close handshake Can we use REST/HTTP/HTTPS persistent connections for this? -Socket is bound to one address for the duration of its lifetime That sounds reasonable, but clients do change IP addresses now and then, so maybe there should be some anticipation of this possibility? -Sockets open sequentially (like current DOS protection in WebSockets) Do you mean their sequence numbers should be strictly increasing incrementally until they roll over? -Cap on number of open sockets per server and total per user agent There was some discussion that people rarely check for the error condition when such caps are exausted, so I'm not sure whether that should be the same as the system cap, or some fraction, or dozens, or a developer configuration parameter.
Re: [whatwg] device element, streams and peer-to-peer connections
On Mon, May 31, 2010 at 7:07 AM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: On Sun, May 30, 2010 at 6:01 PM, James Salsman jsals...@gmail.com wrote: ... With phase vocoder time shifting, UDP delivery as per http://dev.w3.org/html5/html-device/#stream-api would be far superior in quality and intelligibility under packet loss or delay, assuming they went with an open audio codec (or, even better, allowed a choice of speex or ogg vorbis.) ... Where is the discussion regarding phase vocoder time shifting and UDP delivery? I didn't see it in the stream-api section. Some vendors such as Vivox use phase vocoder time shifting to shorten audio segments to catch delayed streams back up to the wall clock. It works even better with TCP or RTP -- http://www.ietf.org/rfc/rfc3550.txt -- as with UDP, because the application layer programmer just has to re-invent the transmission control wheel with UDP anyway.
Re: [whatwg] device element, streams and peer-to-peer connections
On Sat, May 29, 2010 at 4:57 PM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: On Thu, May 27, 2010 at 9:26 PM, James Salsman jsals...@gmail.com wrote: Would it be appropriate to allow selection between reliable delivery involving delay and unreliable delivery with the shorter delay characteristics of UDP by allowing the user to select between the TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input type=file accept=audio as per http://www.w3.org/TR/device-upload and use UDP for synchronous or asynchronous device element I/O? I can see use cases for both methods -- a voice mail, server based application could use a simple form submit upload, but a live voice conferencing app would need real-time access more like in the Capture API that the W3C DAP group has published: http://www.w3.org/TR/capture-api/ It's hard for me to take http://www.w3.org/TR/capture-api/#formatdata seriously. There are no references to open codecs or codec parameters; the only audio codec specified is audio/x-wav, which is a Microsoft-defined union type (RIFF) with a huge number of different possible instance types, including only a few poor quality open vocoders and audio codecs by contemporary performance/bandwidth standards. Where is speex or ogg vorbis? Where are their quality and bit rate parameters? Why is http://www.w3.org/TR/capture-api/#future empty when most of the normative sections say, No exceptions? Where is the compatibility with existing file transfer standards? The security section doesn't contemplate permissions revocation. If audio were segmented into separate files as per http://www.w3.org/TR/capture-api/#captureaudiooptions how would that affect real-time performance on mobile devices? Are these files required to have sequence numbers? With phase vocoder time shifting, UDP delivery as per http://dev.w3.org/html5/html-device/#stream-api would be far superior in quality and intelligibility under packet loss or delay, assuming they went with an open audio codec (or, even better, allowed a choice of speex or ogg vorbis.) Regards, James Salsman
Re: [whatwg] device element, streams and peer-to-peer connections
On Thu, May 27, 2010 at 3:34 PM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: There has been some limited discussion about the peer-to-peer section as it relates to real-time peer to peer gaming applications: http://old.nabble.com/Real-time-networking-in-web-applications-and-games-td27994693.html And references a proposed lower level (UDP-based) WebSocket-like protocol: http://github.com/nardo/torque_sockets/raw/master/TorqueSocket_API.txt The approach here was to punt as much of the flow control/retransmission policy to the application level, whether you're streaming video streams, game data or whatever. Why is relying on TCP for reliable delivery inferior to asking applications to re-implement reliable transmission?
Re: [whatwg] device element, streams and peer-to-peer connections
On Thu, May 27, 2010 at 7:59 PM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: On Thu, May 27, 2010 at 6:22 PM, James Salsman jsals...@gmail.com wrote: Why is relying on TCP for reliable delivery inferior to asking applications to re-implement reliable transmission? In real-time networked applications the retransmission delay imposed by TCP can cause unnecessary and noticeable hitches in the experience; hence many real-time games use custom network stacks built atop UDP Would it be appropriate to allow selection between reliable delivery involving delay and unreliable delivery with the shorter delay characteristics of UDP by allowing the user to select between the TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input type=file accept=audio as per http://www.w3.org/TR/device-upload and use UDP for synchronous or asynchronous device element I/O? Best regards, James Salsman
Re: [whatwg] Speech input element
On Wed, May 19, 2010 at 12:50 AM, Anne van Kesteren ann...@opera.com wrote:
On Tue, 18 May 2010 10:52:53 +0200, Bjorn Bringert bring...@google.com
wrote:
...
Advantages of the speech input element:
- Web app developers do not need to build and maintain a speech
recognition service.
But browser authors would, and it's not clear they will do so in a
cross-platform, compatible way. Client devices with limited cache
memory sizes and battery power aren't very good at the Viterbi beam
search algorithm, which isn't helped much by small caches because it's
mostly random reads across wide memory spans.
- Implementations can have special UI treatment for speech input,
which may be different from that for general audio capture.
I guess I don't really see why this cannot be added on top of the device
element. Maybe it is indeed better though to separate the two. The reason
I'm mostly asking is that one reason we went with device rather than
input is that the result of the user operation is not something that will
partake in form submission
That's not a good reason. Audio files are uploaded with input
type=file all the time, but it wasn't until Flash made it possible
that browser authors started considering the possibilities of
microphone upload, even though they were urged to address the issue a
decade ago:
From: Tim Berners-Lee ti...@w3.org
Date: Fri, 31 Mar 2000 16:37:02 -0500
...
This is a question of getting browser manufacturers to
implement what is already in HTML HTML 4 does already
include a way of requesting audio input. For instance,
you can write:
INPUT name=audiofile1 type=file accept=audio/*
and be prompted for various means of audio input (a recorder,
a mixing desk, a file icon drag and drop receptor, etc).
Here file does not mean from a disk but large body of
data with a MIME type.
As someone who used the NeXT machine's lip service many
years ago I see no reason why browsers should not implement
both audio and video and still capture in this way. There
are many occasions that voice input is valuable. We have speech
recognition systems in the lab, for example, and of course this
is very much needed So you don't need to convince me of
the usefulness.
However, browser writers have not implemented this!
One needs to encourage this feature to be implemented, and
implemented well.
I hope this helps.
Tim Berners-Lee
Further back in January, 2000, that same basic feature request had
been endorsed by more than 150 people, including:
* Michael Swaine - in his article, Sounds like... -
webreview.com/pub/98/08/21/frames - mswa...@swaine.com - well-known
magazine columnist for and long-time editor-in-chief of Dr. Dobb's
Journal
* David Turner and Keith Ross of Institut Eurecom - in their
paper, Asynchronous Audio Conferencing on the Web -
www.eurecom.fr/~turner/papers/aconf/abstract.html -
{turner,ro...@eurecom.fr
* Integrating Speech Technology in Language Learning SIG -
dbs.tay.ac.uk/instil - and InSTIL's ICARE committee, both chaired by
Lt. Col. Stephen LaRocca - gs0...@exmail.usma.army.mil - a language
instructor at the U.S. Military Academy
* Dr. Goh Kawai - g...@kawai.com - a researcher in the fields of
computer aided language instruction and speech recognition, and
InSTIL/ICARE founding member - www.kawai.com/goh
* Ruth Ross - r...@earthlab.com - IEEE Learning Technologies
Standards Committee - www.earthlab.com/RCR
* Phil Siviter - phil.sivi...@brighton.ac.uk - IEEE LTSC -
www.it.bton.ac.uk/staff/pfs/research.htm
* Safia Barikzai - s.barik...@sbu.ac.uk - IEEE LTSC - www.sbu.ac.uk/barikzai
* Gene Haldeman - g...@gene-haldeman.com - Computer Professionals
for Social Responsibility, Ethics Working Group
* Steve Teicher - steve-teic...@att.net - University of Central
Florida; CPSR Education Working Group
* Dr. Melissa Holland - mholl...@arl.mil - team leader for the
U.S. Army Research Laboratory's Language Technology Group
* Tull Jenkins - jenki...@atsc.army.mil - U.S. Army Training
Support Centers
However, W3C decided not to move forward with the implementation
details at http://www.w3.org/TR/device-upload because they were said
to be device dependent, which was completely meaningless, really.
Regards,
James Salsman
[whatwg] forwarded: Google opens VP8 video codec
From: David Gerard dger...@gmail.com Subject: [Wikitech-l] VP8 freed! To: Wikimedia developers, Wikimedia Commons Discussion List http://www.webmproject.org/ http://openvideoalliance.org/2010/05/google-frees-vp8-codec-for-html5-the-webm-project/?l=en http://www.h-online.com/open/news/item/Google-open-source-VP8-as-part-of-the-WebM-Project-1003772.html Container will be .webm, a modified version of Matroshka. Audio is Ogg Vorbis. YouTube is serving up .webm *right now*. Flash will also include .webm.
Re: [whatwg] Speech input element
On Mon, May 17, 2010 at 8:55 AM, Bjorn Bringert bring...@google.com wrote: - What exactly are grammars builtin:dictation and builtin:search? They are intended to be implementation-dependent large language models, for dictation (e.g. e-mail writing) and search queries respectively. I've tried to clarify them a bit in the spec now. There should perhaps be more of these (e.g. builtin:address), maybe with some optional, mapping to builtin:dictation if not available. Bjorn, are you interested in including speech recognition support for pronunciation assessment such as is done by http://englishcentral.com/ , http://www.scilearn.com/products/reading-assistant/ , http://www.eyespeakenglish.com/ , and http://wizworldonline.com/ , http://www.8dworld.com/en/home.html ? Those would require different sorts of language models and grammars such as those described in http://www.springerlink.com/content/l0385t6v425j65h7/ Please let me know your thoughts. Best regards, James Salsman - http://talknicer.com/
[whatwg] syncronous device upload
I propose including the specification at http://www.w3.org/TR/device-upload in HTML5, provided that it is updated to include Speex, Ogg Vorbis, and Ogg Theora. If Google wants to include their codec(s), I hope Ian or one of his colleagues will say so. Because both the Device API and Policy working group and the Web Hypertext Application Technology HTML working group have both been working only on asyncronous device communications as far as I can tell, there is no overlap with that synchronous work done 11 years ago. Would someone at the W3C please replace my updated contact information in http://www.w3.org/TR/device-upload ? I am no longer affiliated with Cisco. My new contact information which I would like included there is: James Salsman device-upl...@talknicer.com or jsals...@gmail.com http://talknicer.com Thank you.
