Re: [whatwg] Device element and the lifetime of the stream objects
Hi Rich, On Wed, Feb 16, 2011 at 6:03 PM, Rich Tibbett ri...@opera.com wrote: We are also trying a different approach but we're not really coming up with anything other than modal dialogs, no-authorization models or policies; none of which are suitable for different reasons. One option that works is, on device click, presenting some kind of async authorization request. However, if we're going to do that then we might as well just implement a Javascript API to call the async authorization request in the first place (in the process saving one user click). I agree that clicking on the device element to bring up an async authorisation request works well because it corresponds strongly to the user's will to start and stop access to the device. However, I think that we should not be trying to save the user a click, because that would risk bothering the user with a dialog before they have made their intention to grant access to the device explicit. The principal application of device is to grant access to webcam devices which, I think, is sensitive enough to warrant the user having to initiate the authorisation process. The device element requires JavaScript to do anything useful. Without Javascript all the authorization interfaces will still work and users will still go through them to authorize access to their device but then...nothing will happen if the page is not running JavaScript. That's not great. There's no fallback or interaction within non-scripted environments so a HTML element seems to be the wrong level for integration. Since the primary use cases for this element are applications that require scripting, I'm not sure that it make sense to worry about device when JS is turned off or unavailable. Indeed, the applications that will use device will probably require JavaScript for other purposes anyway. if JS is disabled, I would expect those applications to render the usual, This page requires JS message. By fallback do you mean fallback content (if the device is unavailable)? If so, doesn't section 3.2.5.1.6 of HTML5: http://dev.w3.org/html5/spec/Overview.html#embedded-content-0 cover this? If the user agent doesn't understand the device tag, it should behave in the same way as an HTML 4 user agent behaves when it encounters any HTML5 element, e.g. video. I think it may be worth revisiting the intention to make device a void element, so that some inner content can be rendered instead (like video). As also mentioned above, in-page UI is harder if we force a particular style for device elements (a button or otherwise) on to developers. We could make UI design easier by supporting a small set of CSS rules to skin the device element. We've been looking at usability and user interface for device vs an async JS API and are finding better ways to make multiple async JS APIs work in our interfaces than we are for device element-based authorizations. It would also be easier for prototyping if we didn't make assumptions on a device element. We can always jump to implementing a device element later on but initially it might be good to prototype in JavaScript with vendor API prefixes in preparation for a final standard approach. The big strength of the device element, just like input type=file, is that it requires a user click to bring up the authorisation dialog. I think that it's a better approach to start with this tried-and-tested security model and fix problems such as skinning, rather than to start with the a Javascript API and add in a security model later. Cheers John On Wed, Feb 16, 2011 at 6:03 PM, Rich Tibbett ri...@opera.com wrote: Andrei Popescu wrote: Hi Anne, On Wed, Feb 16, 2011 at 12:36 PM, Anne van Kesterenann...@opera.com wrote: On Tue, 15 Feb 2011 17:48:24 +0100, Leandro Graciá Gil leandrogra...@chromium.org wrote: All feedback will be greatly appreciated. This is just a thought. Instead of acquiring a Stream object asynchronously there always is one available showing transparent black or some such. E.g. navigator.cameraStream. It also inherits from EventTarget. Then on the Stream object you have methods to request camera access which triggers some asynchronous UI. I thought we were all trying to avoid asynchronous UI (dialogs, infobars, popups, etc), which is a solution that does not scale very well when many different APIs require it. This was one of the main reasons for trying a different approach. We are also trying a different approach but we're not really coming up with anything other than modal dialogs, no-authorization models or policies; none of which are suitable for different reasons. One option that works is, on device click, presenting some kind of async authorization request. However, if we're going to do that then we might as well just implement a Javascript API to call the async authorization request in the first place (in the process saving one user click). Once granted an appropriately named event
Re: [whatwg] Device element and the lifetime of the stream objects
On Wed, Feb 16, 2011 at 2:36 PM, Anne van Kesteren ann...@opera.com wrote: This is just a thought. Instead of acquiring a Stream object asynchronously there always is one available showing transparent black or some such. :) E.g. navigator.cameraStream. It also inherits from EventTarget. Then on the Stream object you have methods to request camera access which triggers some asynchronous UI. Once granted an appropriately named event is dispatched on Stream indicating you now have access to an actual stream. When the user decides it is enough and turns of the camera (or something else happens) some other appropriately named event is dispatched on Stream again turning it transparent black again. This also removes the need for the device element as has been mentioned off-list. Basically, the idea was that device does not really help anyone. It makes custom in-page UI harder, it does not prevent the need for scripting, and it does not help with fallback. Yeah, I'd be quite happy to not have device at all. This is somewhat weird though :-) that is much closer to how i'd want things to work. although from my perspective, i don't know that we need to send many events. I don't see a significant difference between a user with a camera covered by a black tile, or a camera perpetually pointed at a white wall. If the user wants to move where the camera points, that's up to the user, and the app will notice that there's more data streaming. I think it's sufficient for a UA to inform the user that this page is streaming this black region and to allow the user to adjust the virtual camera to point at something else, out through the fourth wall at the user, out the window, at some other device, at a video file, etc.
Re: [whatwg] Device element and the lifetime of the stream objects
Hi Anne, On Wed, Feb 16, 2011 at 12:36 PM, Anne van Kesteren ann...@opera.com wrote: On Tue, 15 Feb 2011 17:48:24 +0100, Leandro Graciá Gil leandrogra...@chromium.org wrote: All feedback will be greatly appreciated. This is just a thought. Instead of acquiring a Stream object asynchronously there always is one available showing transparent black or some such. E.g. navigator.cameraStream. It also inherits from EventTarget. Then on the Stream object you have methods to request camera access which triggers some asynchronous UI. I thought we were all trying to avoid asynchronous UI (dialogs, infobars, popups, etc), which is a solution that does not scale very well when many different APIs require it. This was one of the main reasons for trying a different approach. Once granted an appropriately named event is dispatched on Stream indicating you now have access to an actual stream. When the user decides it is enough and turns of the camera (or something else happens) some other appropriately named event is dispatched on Stream again turning it transparent black again. This also removes the need for the device element as has been mentioned off-list. Basically, the idea was that device does not really help anyone. What do you mean exactly by this? The usecases are pretty clear. It makes custom in-page UI harder, it does not prevent the need for scripting, and it does not help with fallback. I was never under the impression we need to prevent the need for scripting. Why is that a goal? This is somewhat weird though :-) Agreed. And, as I said earlier, I thought the goal was to try something else than asynchronous permission dialogs. What has changed? Thanks, Andrei
Re: [whatwg] Device element and the lifetime of the stream objects
Andrei Popescu wrote: Hi Anne, On Wed, Feb 16, 2011 at 12:36 PM, Anne van Kesterenann...@opera.com wrote: On Tue, 15 Feb 2011 17:48:24 +0100, Leandro Graciá Gil leandrogra...@chromium.org wrote: All feedback will be greatly appreciated. This is just a thought. Instead of acquiring a Stream object asynchronously there always is one available showing transparent black or some such. E.g. navigator.cameraStream. It also inherits from EventTarget. Then on the Stream object you have methods to request camera access which triggers some asynchronous UI. I thought we were all trying to avoid asynchronous UI (dialogs, infobars, popups, etc), which is a solution that does not scale very well when many different APIs require it. This was one of the main reasons for trying a different approach. We are also trying a different approach but we're not really coming up with anything other than modal dialogs, no-authorization models or policies; none of which are suitable for different reasons. One option that works is, on device click, presenting some kind of async authorization request. However, if we're going to do that then we might as well just implement a Javascript API to call the async authorization request in the first place (in the process saving one user click). Once granted an appropriately named event is dispatched on Stream indicating you now have access to an actual stream. When the user decides it is enough and turns of the camera (or something else happens) some other appropriately named event is dispatched on Stream again turning it transparent black again. This also removes the need for thedevice element as has been mentioned off-list. Basically, the idea was thatdevice does not really help anyone. What do you mean exactly by this? The usecases are pretty clear. It makes custom in-page UI harder, it does not prevent the need for scripting, and it does not help with fallback. I was never under the impression we need to prevent the need for scripting. Why is that a goal? The device element requires JavaScript to do anything useful. Without Javascript all the authorization interfaces will still work and users will still go through them to authorize access to their device but then...nothing will happen if the page is not running JavaScript. That's not great. There's no fallback or interaction within non-scripted environments so a HTML element seems to be the wrong level for integration. As also mentioned above, in-page UI is harder if we force a particular style for device elements (a button or otherwise) on to developers. This is somewhat weird though :-) Agreed. And, as I said earlier, I thought the goal was to try something else than asynchronous permission dialogs. What has changed? We've been looking at usability and user interface for device vs an async JS API and are finding better ways to make multiple async JS APIs work in our interfaces than we are for device element-based authorizations. It would also be easier for prototyping if we didn't make assumptions on a device element. We can always jump to implementing a device element later on but initially it might be good to prototype in JavaScript with vendor API prefixes in preparation for a final standard approach. We're doing due diligence on both options so it's worth having this discussion and seeing what other implementers think. Cheers, Rich
[whatwg] Device element and the lifetime of the stream objects
Hi, Looking at the current state of the specification I see there is no mention about the expected lifetime of the stream objects, or to say it in another way, the period in which a page can access the selected device data. We would like to propose that the user can explicitly invalidate an existing stream so that any further access would require a new confirmation by the user. Here is one use case example. Consider a page featuring video chat where the user makes use of the device element to create a Stream. Later, the user finishes chatting and stops using the devices without leaving the page. At this point, we need to prevent a malicious Web application from accessing the device camera or microphone again without the user's consent. Given the above case, we don't think that the lifetime of the Stream objects should be controlled exclusively by the Web application. We think that the specification should state the UA must allow the user to explicitly revoke access. The API should then be then extended with a way to handle gracefully the situation when access is revoked in the middle of a streaming session. All feedback will be greatly appreciated. Thanks, Leandro
Re: [whatwg] Device element and the lifetime of the stream objects
2011/2/15 Leandro Graciá Gil leandrogra...@chromium.org: Given the above case, we don't think that the lifetime of the Stream objects should be controlled exclusively by the Web application. We think that the specification should state the UA must allow the user to explicitly revoke access. The API should then be then extended with a way to handle gracefully the situation when access is revoked in the middle of a streaming session. All feedback will be greatly appreciated. As long as UAs can replace the stream with a stream of compressed solid black, i'm happy. I believe such a stream could probably have very few frames (hopefully 2?). With regard to being able to resume the stream, I think that's something UAs could do w/o involving web apps. For the case where the web app wants to turn off the stream, it's ok to let the app ask to turn the stream back on, but otherwise, I don't think I want a web app to be able to bother the user if the user has chosen to turn mute the stream.
Re: [whatwg] Device Element
On Sun, 9 Jan 2011 21:10:58 +, Bjartur Thorlacius wrote: On 1/9/11, Glenn Maynard gl...@zewt.org wrote: File access control is currently, very clearly and very deliberately, handled by the browser: web pages can only access files the user gives to the page by selecting them in form input boxes. What you're actually saying is that this should be removed, web pages should be able to access any local file that the OS user account the script is running as has access to, and that users should control what files they want web pages to access by modifying the operating system's ACL's to grant and revoke access to web pages. Precisely. Any hurdles I've foreseen with that method so far are OS' faults. This is way too pragmatic and useless. That is a security breach of the current desktop apps. My GTD app should never ever read my invoicing documents. A web app should never have access to all my stuff. That's why I said kernel ACLs should never be confused with this. The kind of ACL we're talking is just like geolocation, js execution, cookies, ... a page access X device because a user say yes to a warning dialog and choose X device. Nothing more. Please stop CC'ing to me, I'm on the list.
Re: [whatwg] Device Element
On 1/9/11, Glenn Maynard gl...@zewt.org wrote: Access control is tied closely to the nearby discussion on privilege escalation (which came out of this thread). I do believe it's the browser's job to decide what access a particular page gets, and to give the user control over privilege escalation. I hate the analogy, but the browser is to web apps what the operating system is to native apps: from providing the APIs to let them do things, to enforcing permissions. The difference, of course, is that the default trust level for remote apps is much lower than for native apps. It seems to me that the difference can be implemented by running remote apps with lower privileges. No need to reimplement all access control mechanism. The wheel does exist, already. If you want *all* of the OS running on top of the OS, you should host Vita Nuova Inferno. Browser access control shouldn't be as complex as OS permissions, but it does need to exist, and I do believe it'll need to be fleshed out more thoroughly than it is currently for web apps to be a real alternative to native for many use cases, and for others, to make them less acutely second-class. As I mentioned in the other thread, privilege escalation ranges from relaxing nuisance-preventing restrictions (context menu cancellation, browser fullscreen, opening windows) to high-trust operations like broader local file access and yes, device access. So, OS permissions are too complex, so you figure it's best to build another permission system on top of the existing ones? Why? The nuisance-prevention may be implemented by browsers, but privilege escalation to e.g. broader file access is clearly the OS' job. (Not to endorse device access in general--that request for low-level access to Bluetooth devices is, frankly, crazy--but a serial API isn't unreasonable at all.) Seems to be about the only point we agree on. The particular mechanisms to handle this should be up to the browser, of course, but I firmly believe it's the browser's job. I strongly disagree. Running applications written in other programming languages than JavaScript with limited privileges may be useful to a user. Thus permissions should in no way be tied to browsers.
Re: [whatwg] Device Element
On Sun, Jan 9, 2011 at 7:21 AM, Bjartur Thorlacius svartma...@gmail.com wrote: So, OS permissions are too complex, so you figure it's best to build another permission system on top of the existing ones? Why? 1) OS permissions are not adequately standardized. Every OS has its own permissions model. This makes portability hard, and increases the risk of security bugs caused by the same policy being enforced differently by different OSes. 2) Conventional OS permissions are based on the idea of protecting different users from each other, not protecting users from their own programs. It's assumed that users do not run any program unless they trust it fully. This is not useful in the case of web apps, where the app is a web page that we assume is totally untrusted. OS permissions are coming around to the idea of untrusted apps, but only slowly and (see point 1) inconsistently. Regardless, this point was settled well over a decade ago. Web pages have their own security model, on top of system permissions. This was true as soon as anyone implemented scripting for web pages, since web page scripts have always been sandboxed at a lower permission level than any full program. There's no point in talking about it.
Re: [whatwg] Device Element
On 1/9/11, Aryeh Gregor simetrical+...@gmail.com wrote: On Sun, Jan 9, 2011 at 7:21 AM, Bjartur Thorlacius svartma...@gmail.com wrote: So, OS permissions are too complex, so you figure it's best to build another permission system on top of the existing ones? Why? 1) OS permissions are not adequately standardized. Every OS has its own permissions model. This makes portability hard, and increases the risk of security bugs caused by the same policy being enforced differently by different OSes. Applications need not be aware of the exact permission model used. As you stated yourself, OS are implementing security models accounting for untrusted apps. I find it unnecessary to duplicate the maintenance work.
Re: [whatwg] Device Element
On 1/9/11, Glenn Maynard gl...@zewt.org wrote: File access control is currently, very clearly and very deliberately, handled by the browser: web pages can only access files the user gives to the page by selecting them in form input boxes. What you're actually saying is that this should be removed, web pages should be able to access any local file that the OS user account the script is running as has access to, and that users should control what files they want web pages to access by modifying the operating system's ACL's to grant and revoke access to web pages. Precisely. Any hurdles I've foreseen with that method so far are OS' faults.
Re: [whatwg] Device Element
On Sun, Jan 9, 2011 at 4:10 PM, Bjartur Thorlacius svartma...@gmail.com wrote: On 1/9/11, Glenn Maynard gl...@zewt.org wrote: File access control is currently, very clearly and very deliberately, handled by the browser: web pages can only access files the user gives to the page by selecting them in form input boxes. What you're actually saying is that this should be removed, web pages should be able to access any local file that the OS user account the script is running as has access to, and that users should control what files they want web pages to access by modifying the operating system's ACL's to grant and revoke access to web pages. Precisely. Any hurdles I've foreseen with that method so far are OS' faults. Browsers can use OS mechanisms *under the hood* if they want, but it's absolutely the browser's job to expose website-specific privilege escalation to the user, and it's absolutely the job of designers of web APIs to design mechanisms to request it. (For example, a file input box is a form of requesting privilege escalation. By selecting a file, you elevate the page's permissions temporarily to give it access to that file.) I'm very certain that HTML5 file access will not be changed to allow any script to access any file that the OS allows the Javascript engine to access. -- Glenn Maynard
Re: [whatwg] Device Element
On Sun, Jan 9, 2011 at 7:21 AM, Bjartur Thorlacius svartma...@gmail.com wrote: So, OS permissions are too complex, so you figure it's best to build another permission system on top of the existing ones? Why? The nuisance-prevention may be implemented by browsers, but privilege escalation to e.g. broader file access is clearly the OS' job. File access control is currently, very clearly and very deliberately, handled by the browser: web pages can only access files the user gives to the page by selecting them in form input boxes. What you're actually saying is that this should be removed, web pages should be able to access any local file that the OS user account the script is running as has access to, and that users should control what files they want web pages to access by modifying the operating system's ACL's to grant and revoke access to web pages. -- Glenn Maynard
Re: [whatwg] Device Element
I hope I was clear enough now. No kernel ACLs and stuff like that. What's wrong with access control?
Re: [whatwg] Device Element
On Sat, 8 Jan 2011 12:24:08 +, Bjartur Thorlacius wrote: I hope I was clear enough now. No kernel ACLs and stuff like that. What's wrong with access control? With access control.. nothing. With kernel ACLs.. everything. That is out of scope here.
Re: [whatwg] Device Element
On 1/8/11, drese...@thinkdigital.pt drese...@thinkdigital.pt wrote: On Sat, 8 Jan 2011 12:24:08 +, Bjartur Thorlacius wrote: What's wrong with access control? With access control.. nothing. With kernel ACLs.. everything. That is out of scope here. The system provides access control. What I don't understand is what you've got against it. I'm not saying ACL must be implemented in kernel. Run them in Ring 1, userspace or wherever you want to. They're clearly out of scope of the WHATWG.
Re: [whatwg] Device Element
On Sat, Jan 8, 2011 at 6:50 PM, Bjartur Thorlacius svartma...@gmail.com wrote: The system provides access control. I'm not sure what you're saying here. The browser provides access control for web applications. This is already true in lots of places -- like geolocation and network access, to name recent ones. Even if all native applications can access particular hardware as far as the system is concerned, it certainly doesn't follow that the browser can expose the same privilege to web applications.
Re: [whatwg] Device Element
On 1/8/11, Aryeh Gregor simetrical+...@gmail.com wrote: On Sat, Jan 8, 2011 at 6:50 PM, Bjartur Thorlacius svartma...@gmail.com wrote: The system provides access control. I'm not sure what you're saying here. The browser provides access control for web applications. This is already true in lots of places -- like geolocation and network access, to name recent ones. Even if all native applications can access particular hardware as far as the system is concerned, it certainly doesn't follow that the browser can expose the same privilege to web applications. When an user application requests screen access, I don't grant it carte blanche access to a screen of mine. Then I would need a screen per graphical app. Rather, I partition my screen and, given it has the necessary privileges, grant it revocable access to a frame. Then, I revoke the access and free up the frame for another application. Similar with storage space and networking. I don't see any reason for language implementations (such as JavaScript VMs) to reimplement hardware multiplexing and access granting, forcing system administrators (i.e. users) to manage access controls for 'normal' apps on one hand, but grant the JavaScript VM administrative privileges and then configure 'WebApps' seperately. As a side note, I don't remember how the topic morphed into what it's now, you'd think that all of this would be an implementation detail. The JavaScript application could be a well audited and trusted system administration application, for all we know.
Re: [whatwg] Device Element
On Sat, Jan 8, 2011 at 6:53 PM, Aryeh Gregor simetrical+...@gmail.com wrote: On Sat, Jan 8, 2011 at 6:50 PM, Bjartur Thorlacius svartma...@gmail.com wrote: The system provides access control. I'm not sure what you're saying here. The browser provides access control for web applications. This is already true in lots of places -- like geolocation and network access, to name recent ones. Even if all native applications can access particular hardware as far as the system is concerned, it certainly doesn't follow that the browser can expose the same privilege to web applications. Access control is tied closely to the nearby discussion on privilege escalation (which came out of this thread). I do believe it's the browser's job to decide what access a particular page gets, and to give the user control over privilege escalation. I hate the analogy, but the browser is to web apps what the operating system is to native apps: from providing the APIs to let them do things, to enforcing permissions. The difference, of course, is that the default trust level for remote apps is much lower than for native apps. Browser access control shouldn't be as complex as OS permissions, but it does need to exist, and I do believe it'll need to be fleshed out more thoroughly than it is currently for web apps to be a real alternative to native for many use cases, and for others, to make them less acutely second-class. As I mentioned in the other thread, privilege escalation ranges from relaxing nuisance-preventing restrictions (context menu cancellation, browser fullscreen, opening windows) to high-trust operations like broader local file access and yes, device access. (Not to endorse device access in general--that request for low-level access to Bluetooth devices is, frankly, crazy--but a serial API isn't unreasonable at all.) The particular mechanisms to handle this should be up to the browser, of course, but I firmly believe it's the browser's job. -- Glenn Maynard
Re: [whatwg] Device Element
On Tue, 4 Jan 2011 22:09:20 +, Bjartur Thorlacius wrote: On 1/4/11, Diogo Resende drese...@thinkdigital.pt wrote: Flash is insecure because there's no security policies. It's similiar to the firefox feature to read files: you read all or you read none. That's not a good policy. Something similar to the geolocation would be better (this specific site/app can access this specific device). Maybe I was not clear. Example: - User goes to an app, clicks on a button - App requests a serial device access for a Meteorology Device (this should really be a string or something like that, not a device category or class) - Browser notifies user, shows list of devices for the user to pick ONE or to deny (in the middle of this a sudo-like box might appear, I really don't care about it) - App gets access to a limited serial API just to that device (or any other that a user might already given access) I hope I was clear enough now. No kernel ACLs and stuff like that.
Re: [whatwg] Device Element
Flash is insecure, so HTML5 should be too? Seriously? Flash is insecure because there's no security policies. It's similiar to the firefox feature to read files: you read all or you read none. That's not a good policy. Something similar to the geolocation would be better (this specific site/app can access this specific device).
Re: [whatwg] Device Element
On 1/4/11 5:48 AM, Diogo Resende wrote: Flash is insecure because there's no security policies. It's similiar to the firefox feature to read files: you read all or you read none. That's not a good policy. Something similar to the geolocation would be better (this specific site/app can access this specific device). The problem with adding more capabilities like this in an ad-hoc way is that it involves user trust, and worse yet it involves trust in things the user can't audit and won't realize they're trusting. For example, say www.foo.com requests access to the user's USB devices. If the user allows the request, then they are trusting that: 1) The site is not malicious (this is the part the user probably thinks about when deciding to trust). 2) The site is loaded securely (entirely over https:). If not, there's no guarantee you're talking to the right site. 3) The site has no script-injection vulnerabilities. 4) The site won't be hacked. 5) All the user's CAs are aboveboard and not cooperating with the ISP to fake sites (not a given in some countries!). There are likely a few other things being trusted here that I'm not thinking of; I can guarantee that typical users won't think of #3-5 above, and many won't think of #2 above. I realize that _you_ trust #2-4 about your own web site. But frankly, history says I shouldn't thus trust your site Perhaps we need a stronger model where permission to access devices is granted not to an origin but to a particular script (with the hash of the script stored and permission denied on hash mismatch or something). I don't know. But granting blanket access to an entire origin seems questionable to me. -Boris
Re: [whatwg] Device Element
On 1/4/11, Diogo Resende drese...@thinkdigital.pt wrote: Flash is insecure because there's no security policies. It's similiar to the firefox feature to read files: you read all or you read none. That's not a good policy. Something similar to the geolocation would be better (this specific site/app can access this specific device). File ACLs are usually implemented in kernels, not web browsers. I'll take ACLs in userspace, but you can surely find a more appropriate place to implement them.
Re: [whatwg] Device element
That was the point of what I said in the previous message. The default security level wouldn't prompt users every time a script changes. But for users who are interested and do understand what's going on--they have the option of checking to see if a script has changed and can do some more verification. On Tue, Jan 4, 2011 at 7:15 PM, Glenn Maynard gl...@zewt.org wrote: And: it still doesn't help. Asking a user whether changes to a Javascript file are okay is meaningless. Regular users don't know Javascript; there's no way they can know whether to accept a change or not. No general security model can be built around requiring the user to understand the technical issues behind the security.
Re: [whatwg] Device Element
On Mon, Jan 3, 2011 at 8:12 AM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/2/11 2:51 PM, Diego Perini wrote: Of course this require file security privileges (security.fileuri.strict_origin_policy = true) You mean false. Yes, need to set that to false to have those privileges. That option is there for testing purposes only. If you're asking users to change them, you're asking them to leave their browser exploitable. Please don't do that. I am switching that bit on/off when I need it (and the RS232/USB device is connected), I am not asking others to do it. I hope this feature/capability does not disappear without alternatives in next versions of Firefox (or next time I will just have to shut-up). Mostly I am looking for other browser to implement this serial access like Firefox does, with security as one of the top objectives. This also means limit what other plug-ins can currently do on user machines ! For example can Flash overcome this restriction and access the devices through the file: protocol ? -- Diego -Boris
Re: [whatwg] Device Element
On 1/3/11 7:47 AM, Diego Perini wrote: I am switching that bit on/off when I need it (and the RS232/USB device is connected), I am not asking others to do it. So while it's off, the user is vulnerable. Is the software doing this a Firefox extension? Mostly I am looking for other browser to implement this serial access like Firefox does Firefox doesn't implement serial access. It just reads files. What's exposed as files on a Unix system can be ... extensive (witness /proc on Linux, say). It's up to the kernel. This also means limit what other plug-ins can currently do on user machines Plug-ins are running native code. They can't be limited unless you sandbox the process they're in completely (which typically breaks plug-ins; talk to Google about the pains they had to go to to sandbox Flash, even _with_ Adobe's cooperation). For example can Flash overcome this restriction and access the devices through the file: protocol ? Of course, if the kernel exposes them as files. -Boris
Re: [whatwg] Device Element
On Mon, Jan 3, 2011 at 7:47 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/3/11 7:47 AM, Diego Perini wrote: I am switching that bit on/off when I need it (and the RS232/USB device is connected), I am not asking others to do it. So while it's off, the user is vulnerable. Is the software doing this a Firefox extension? No no... haven't tried that route but I understand it is another possibility. I don't remember having said I use this bit for some real application, just testing. Mostly I am looking for other browser to implement this serial access like Firefox does Firefox doesn't implement serial access. It just reads files. What's exposed as files on a Unix system can be ... extensive (witness /proc on Linux, say). It's up to the kernel. You are correct, Firefox doesn't implement serial access by itself, it just let me use the OS directly (if security configuration restrictions are removed). So let's say I like the functionality this happens to give me with RS232/USB devices. Even with those restrictions in place. This is why I agree with the request for a device interface specification (in HTML5) that will make this inter-operable. This also means limit what other plug-ins can currently do on user machines Plug-ins are running native code. They can't be limited unless you sandbox the process they're in completely (which typically breaks plug-ins; talk to Google about the pains they had to go to to sandbox Flash, even _with_ Adobe's cooperation). For example can Flash overcome this restriction and access the devices through the file: protocol ? Of course, if the kernel exposes them as files. So next question is why allow Adobe Flash and plug-ins in general to do that wildly and not allow others to have the same capability and be so paranoid about security when that is already broken by other means at higher levels ? -- Diego -Boris
Re: [whatwg] Device Element
On 1/3/11, Boris Zbarsky bzbar...@mit.edu wrote: On 1/3/11 7:47 AM, Diego Perini wrote: Mostly I am looking for other browser to implement this serial access like Firefox does Firefox doesn't implement serial access. It just reads files. What's exposed as files on a Unix system can be ... extensive (witness /proc on Linux, say). It's up to the kernel. This should be leveraged. Standardized API for file access and standardized file names are a powerful combination. Sysadmins/users can block or grant access to files, without even informing the browser of the criteria or how requests are evaluated. For example can Flash overcome this restriction and access the devices through the file: protocol ? Of course, if the kernel exposes them as files. Or a userspace abstractions, such as KIO. For those with a KDE bent of mind.
Re: [whatwg] Device Element
On Mon, Jan 3, 2011 at 4:45 PM, Diego Perini diego.per...@gmail.com wrote: You are correct, Firefox doesn't implement serial access by itself, it just let me use the OS directly (if security configuration restrictions are removed). Please remember that HTML5 file access does not give you a general-purpose serial API, regardless of whether you can access the serial device. Accessing block devices (including regular files) is different from accessing character devices (eg. serial devices). HTML5 file APIs assume files have properties of block devices (seekable, with a meaningful size), and don't provide any of the critical APIs for character devices: true nonblocking reads (FileReader does async reads, which aren't the same thing); notification when data is available (again, not the same as FileReader's onprogress event). And that's ignoring all of the important nitty details of serial devices, like port speed and other parameters. I suspect you're hitting far more happens-to-work with what you're doing than you realize. So next question is why allow Adobe Flash and plug-ins in general to do that wildly and not allow others to have the same capability and be so paranoid about security when that is already broken by other means at higher levels ? Flash is insecure, so HTML5 should be too? Seriously? -- Glenn Maynard
Re: [whatwg] Device Element
On Mon, Jan 3, 2011 at 4:45 PM, Diego Perini diego.per...@gmail.com wrote: So next question is why allow Adobe Flash and plug-ins in general to do that wildly and not allow others to have the same capability and be so paranoid about security when that is already broken by other means at higher levels ? If something is available in HTML, any website can run it. Websites can't just install plugins. The only way for them to install plugins is by getting the user to run an executable, which browsers put lots of scary be careful signs around just like for any other downloaded executable. If the website can get the user to install a plugin, it can get them to run any program, so there's no security at all from that point forward. Yes, HTML features can require user confirmation, like geolocation usually does. But the kind of mild confirmation needed for geolocation is not suitable for allowing arbitrary file read/write access. The latter would require much scarier-looking permission, in fact about as much as installing a program, since it would be trivial to escalate to arbitrary code execution. We don't want to encourage websites to pop up warnings like that all the time, because 1) many users won't give permission, which makes the feature much less useful to authors; and 2) it desensitizes users so they click yes all the time. So this is really apples and oranges. Part of the point of the web is it's relatively safe. You can't compare web platform features to arbitrary code like plugins. This sort of feature will probably come in time, driven by Chrome OS if nothing else, but it will have to be thought out carefully to balance security against usability.
Re: [whatwg] Device Element
On Mon, Jan 3, 2011 at 11:28 PM, Glenn Maynard gl...@zewt.org wrote: On Mon, Jan 3, 2011 at 4:45 PM, Diego Perini diego.per...@gmail.com wrote: You are correct, Firefox doesn't implement serial access by itself, it just let me use the OS directly (if security configuration restrictions are removed). Please remember that HTML5 file access does not give you a general-purpose serial API, regardless of whether you can access the serial device. Accessing block devices (including regular files) is different from accessing character devices (eg. serial devices). True but in case of character devices which continuously send a string of max 1Kb (or less) this is not a blocking factor. I agree that having a full set of API to handle b/c devices would be everybody dream but we all know we have to work around some limitations, it has always been so. HTML5 file APIs assume files have properties of block devices (seekable, with a meaningful size), and don't provide any of the critical APIs for character devices: true nonblocking reads (FileReader does async reads, which aren't the same thing); notification when data is available (again, not the same as Polling for changes was enough to get the results but some sort of mutation events would have worked better (depending on device output). Currently the GPS device I am using only output text but it could as well generate better data for parsing, like XML or your preferred one. FileReader's onprogress event). And that's ignoring all of the important nitty details of serial devices, like port speed and other parameters. Setting the device serial configuration can be done outside the browser (until a proper specification and API exists). Maybe having those settings in the about:config would allow access more easily to those not willing to open the terminal. Adding strong security will not diminish the importance of having these capabilities in new OS centric browsers. I suspect you're hitting far more happens-to-work with what you're doing than you realize. Yeah, maybe. However if you had payed attention to the discussion I am just testing :) I repeatedly stated that I am OK with current level of security and I wish better and more secure methods are found and implemented (if needed). I added that I agree with the proposal to find alternatives that will allow to make easy for browsers to communicate with RS232/USB devices securely. So next question is why allow Adobe Flash and plug-ins in general to do that wildly and not allow others to have the same capability and be so paranoid about security when that is already broken by other means at higher levels ? Flash is insecure, so HTML5 should be too? Seriously? Please read it again I only asked why ? I said Flash is insecure yes, the rest are your words (purposely misinterpreted)... HTML5 should be secure but I am not sure allowing it to run side by side with insecure (and evil) content makes that a reality :-) The Device element proposal and the parts on security that Seth put forth in previous messages seemed a good start to me, that's all. Have fun, -- Diego -- Glenn Maynard
Re: [whatwg] Device Element
On Wed, Dec 29, 2010 at 9:53 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 12/29/10 6:48 AM, Diego Perini wrote: Hmmm... I can currently read incoming RS232 and USB data in my OS X using Firefox 3.6.13 and the file: protocol: file:///dev/tty.BT-GPS010B62-SerialPort do I have an insecure system ? An insecure browser ? And _this_ sort of thing is why Firefox and other browsers don't ever allow an http:// page to link to a file:// URL, read data from a file:// URL, or otherwise interact with a file:// URL. Boris, don't get confused by the questions in my message above, I really don' feel insecure for that existing option in Firefox. It was a tentative answer to so many security concerns and I wanted to tell that what they were after is almost already available in Firefox/UNIX. I feel it is a great and unique feature of Firefox, no other browser (that I am aware of) implement that level of OS integration (that maybe only for UNIX systems though). Of course this require file security privileges (security.fileuri.strict_origin_policy = true) but having to open about:config or having to consent through a button is not a problem for what I need it and probably what in general is needed. I believe the process Seth described in his message would be more than enough for the security problems this poses (relying on user consent is what plug-ins also do). -- Diego -Boris
Re: [whatwg] Device Element
On 1/2/11 2:51 PM, Diego Perini wrote: Of course this require file security privileges (security.fileuri.strict_origin_policy = true) You mean false. That option is there for testing purposes only. If you're asking users to change them, you're asking them to leave their browser exploitable. Please don't do that. -Boris
Re: [whatwg] Device Element
Hmmm... I can currently read incoming RS232 and USB data in my OS X using Firefox 3.6.13 and the file: protocol: file:///dev/tty.BT-GPS010B62-SerialPort do I have an insecure system ? An insecure browser ? Or do we already have part of what is requested in some browser ? -- Diego On Wed, Dec 29, 2010 at 2:21 AM, Seth Brown lear...@gmail.com wrote: Sorry I meant potentially has that ability. Any software you install could do this without you knowing it. Sent from my iPhone On Dec 28, 2010, at 8:12 PM, Bjartur Thorlacius svartma...@gmail.com wrote: Any software that you install has the ability to jump in and take control of RS232 devices. That's news to me. What if my terminal, HTML and PDF reader don't have access to every device attached to my system, any more than my MTA?
Re: [whatwg] Device Element
Never thought about doing it that way. Since unix handles devices as files it looks like implementing this on unix and unix like systems would be relatively easy. On Wed, Dec 29, 2010 at 9:48 AM, Diego Perini diego.per...@gmail.com wrote: Hmmm... I can currently read incoming RS232 and USB data in my OS X using Firefox 3.6.13 and the file: protocol: file:///dev/tty.BT-GPS010B62-SerialPort do I have an insecure system ? An insecure browser ? Or do we already have part of what is requested in some browser ? -- Diego
Re: [whatwg] Device Element
On Wed, Dec 29, 2010 at 6:48 AM, Diego Perini diego.per...@gmail.com wrote: Hmmm... I can currently read incoming RS232 and USB data in my OS X using Firefox 3.6.13 and the file: protocol: file:///dev/tty.BT-GPS010B62-SerialPort do I have an insecure system ? An insecure browser ? Can you do that from a webpage served from a http server? Can you do that from a HTML file saved to your desktop? Can you do that from a HTML file saved anywhere else on your filesystem? For firefox, the answer should be no in all three cases, unless you've changed security related settings. Thus the answer to your questions should be no. Definitely file a bug on Firefox (bugzilla.mozilla.org) if that isn't the case. / Jonas
Re: [whatwg] Device Element
unless you've changed security related settings. How is that any different from the proposal? Accessing the device tag should require user consent--the same as changing security related settings. On Wed, Dec 29, 2010 at 12:04 PM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Dec 29, 2010 at 6:48 AM, Diego Perini diego.per...@gmail.com wrote: Hmmm... I can currently read incoming RS232 and USB data in my OS X using Firefox 3.6.13 and the file: protocol: file:///dev/tty.BT-GPS010B62-SerialPort do I have an insecure system ? An insecure browser ? Can you do that from a webpage served from a http server? Can you do that from a HTML file saved to your desktop? Can you do that from a HTML file saved anywhere else on your filesystem? For firefox, the answer should be no in all three cases, unless you've changed security related settings. Thus the answer to your questions should be no. Definitely file a bug on Firefox (bugzilla.mozilla.org) if that isn't the case. / Jonas
Re: [whatwg] Device Element
On Wed, 29 Dec 2010 01:44:09 +0100, Seth Brown lear...@gmail.com wrote: First I'd like to point out that accessing hardware is already possible through ActiveX, and I think I've heard of some people doing it with XUL. I would like to standardize these hardware interactions. I don't think it should be done without strict security measures, Indeed. This matters... Also it should be possible to allow an administrator to set policies to block the browser from accessing usb/RS232. So does this... At some point it is going to be necessary for the web apps to interact with hardware I hate the concept of platform specific apps and I think that once HTML5 if fully implemented apps will start to disappear. The only real hurdle left is hardware interaction, but it will happen at some point. Why don't we make sure it gets done right? Indeed. This is the point of initiatives like Opera platform, OMTP's BONDI, and WAC [1]. I'm going to start working on a concrete proposal. I've never written one before, so any advice would be appreciated -- should I just look through other completed specs and format it accordingly? That helps in making the content easy to find and understand for people who are used to reading specs. I suggest you look at W3C's DAP group [2] for an idea of what people are doing - and at WAC for an idea of stuff that is being shipped already. [1] http://www.jil.org/web/jil or http://en.wikipedia.org/wiki/Wholesale_Application_Community [2] http://www.w3.org/2009/dap/ cheers Chaals -- Charles McCathieNevile Opera Software, Standards Group je parle français -- hablo español -- jeg lærer norsk http://my.opera.com/chaals Try Opera: http://www.opera.com
Re: [whatwg] Device Element
I agree with Seth and the others that promote the spec idea. If a detailed spec is needed although I too do not have much experience in writing them I would definitely give some of my time along with Seth to writing one. The thing with html5 is how long the spec is going to be in spec status and even then wide implementation is still quite a bit of time away. If this is not going to be implemented you will have to either use flash, or native code, or activeX whoever knows if that XUL comment is true but if it is then it's the same problem of no standard. Heck svg just this year started becoming standard with new browsers. Considering google has native client does that mean in the future all I have a choice over is flash + native client (google and all desktops) or native code (all others) and based on the platform design code natively? Sounds like the current times problem to me. Anyone else coding a web application with camera/microphone access for android, iOS, Winphone 7, blackberry, Mac and PC? Each with their own flaws to do the same darn thing :(. It would be a lot easier if this was the same don't you think? Hope someone else out there maybe with experience can usher some guidance and let me and Seth know how we can help with this spec? Kind regards, Nikita On Wednesday, December 29, 2010, Charles McCathieNevile cha...@opera.com wrote: On Wed, 29 Dec 2010 01:44:09 +0100, Seth Brown lear...@gmail.com wrote: First I'd like to point out that accessing hardware is already possible through ActiveX, and I think I've heard of some people doing it with XUL. I would like to standardize these hardware interactions. I don't think it should be done without strict security measures, Indeed. This matters... Also it should be possible to allow an administrator to set policies to block the browser from accessing usb/RS232. So does this... At some point it is going to be necessary for the web apps to interact with hardware I hate the concept of platform specific apps and I think that once HTML5 if fully implemented apps will start to disappear. The only real hurdle left is hardware interaction, but it will happen at some point. Why don't we make sure it gets done right? Indeed. This is the point of initiatives like Opera platform, OMTP's BONDI, and WAC [1]. I'm going to start working on a concrete proposal. I've never written one before, so any advice would be appreciated -- should I just look through other completed specs and format it accordingly? That helps in making the content easy to find and understand for people who are used to reading specs. I suggest you look at W3C's DAP group [2] for an idea of what people are doing - and at WAC for an idea of stuff that is being shipped already. [1] http://www.jil.org/web/jil or http://en.wikipedia.org/wiki/Wholesale_Application_Community [2] http://www.w3.org/2009/dap/ cheers Chaals -- Charles McCathieNevile Opera Software, Standards Group je parle français -- hablo español -- jeg lærer norsk http://my.opera.com/chaals Try Opera: http://www.opera.com -- Nikita Eelen Network Manager AVE INTERVISION LLC 6 4 0 West State Street Alliance, Ohio 4 4 6 0 1 Office: (800) 448-9126 ext. 101 Cell: 330-257-9319 email: nee...@amvonet.com
Re: [whatwg] Device Element
On 12/29/10 6:48 AM, Diego Perini wrote: Hmmm... I can currently read incoming RS232 and USB data in my OS X using Firefox 3.6.13 and the file: protocol: file:///dev/tty.BT-GPS010B62-SerialPort do I have an insecure system ? An insecure browser ? And _this_ sort of thing is why Firefox and other browsers don't ever allow an http:// page to link to a file:// URL, read data from a file:// URL, or otherwise interact with a file:// URL. -Boris
Re: [whatwg] Device Element
How about making a concrete proposal as to what it should look like? If Google was to implement it and turn it into a concrete proposal, I wouldn't have a problem with it either. As it is right now the spec for usb/RS232 is useless IMHO. Silvia. On Tue, Dec 28, 2010 at 4:02 AM, Diego Perini diego.per...@gmail.com wrote: On Tue, Dec 28, 2010 at 3:16 AM, Seth Brown lear...@gmail.com wrote: I saw on the spec page that you were considering limiting the device element to audio/visual streams. I thinks this would be a mistake. I'm currently working on integrating serial port connected hardware with a web application I'm developing. The only solution is for the user to install local adapter software. This defeats the purpose of using a web app over desktop software. In order for web applications to be gain traction over desktop software, they must be able to interface with usb/RS232. I believe the security trade off is worth it. I also believe that the working group should make the device element spec a high priority. If you don't google will probably implement their own version for chrome OS(it will be necessary in a browser based OS model). Thanks, Seth Brown Agreed. Access to RS232 data from scripts will promote the use of web technologies in new applications for the embedded market. -- Diego
Re: [whatwg] Device Element
On 2010-12-28 09:53, Silvia Pfeiffer wrote: How about making a concrete proposal as to what it should look like? If Google was to implement it and turn it into a concrete proposal, I wouldn't have a problem with it either. As it is right now the spec for usb/RS232 is useless IMHO. Silvia. Yeah! And not to mention the security bomb. I don't like the idea of a web app accessing my USB stick without my permission. So that means that all browsers would need to ask the user for permission. That is at the Browser level. Then there is the OS and/or driver level which all those wanting this in this list so far is forgetting. OS priviledge levels. An administrator (school, work, library, fire/police/hospital, or home network) might have set the OS to not allow a regular user to access say a USB stick or other USB or serial device with their regular user account. Windows Vista+, Mac OS X, and Linux does this. Unless something was blocked by an admin, then anything available in usermode is available to anything else in usermode. So if all browsers supported an arbitrary USB/serial device API like maybe get device config, set device config, read data, write data, those are the basics right? (any more than that and it's no longer generic) And they would also need to allow the user to explicitly enable which device should be exposed. Maybe the browser could when asked by a web app to access a device, simply show a prompt informing the user that this/that app wants device access, then show all devices the OS presents (Readable/Writable state etc. admin disabled ones are not listed etc.) in a list unless the web app asked for a specific device, in which case only list the matches or exact match. If the user allows the webapp access, then and only then does the webapp get access to the device/or devices the user specified. The webapp should also be marked as being secure (HTTPS) or not, and the user should be able to set the browser to ignore non-secure webapps (HTTP) etc (can webapps be signed with a certificate at all?). Don't get me wrong, I understand those of you advocating so hard for this in the list, but the issue is that weather data and test stats although similar are different enough that a generic API is needed, and a generic API needs a lot of security precautions as I'm sure many here may have a USB harddrive hooked up to the system, the last thing you want is for some webapp that seems like it's just some microphone voice FX toy suddenly barge through your harddrive right? Or worse, that weather app starts poking around your microphone, or webcam. A lot of people has certain devices hooked up, since USB is so versatile there is anything from: recording devices (mic, cam, etc) to output devices (speakers, headphones, mini displays/embed keyboard screens, picture frames etc), to networking (network cards/routers, controllers for household electrics), and who knows what else. So the remark someone made that the security trade is worth it? Nah-ah. Nothing on the net should ever have direct access to any input/output/storage device or similar at all. Any webapp (I use webapp as I consider HTML, Java, Flash in the same boat in this particular issue) should go through 3 layers of security. The Browser layer (the listing/prompt I described above), The User layer (if the OS supports it, let the user dictate which software can use which devices), and then the OS layer (admin settings, intranet, driver config etc). I know some people here are drooling at the idea of driverless USB devices that a webapp talks to directly, but it's never going to happen. The OS (or admin configuration) still control which devices are available, even if they are HID. And no browser would allow blind access to the OS's devices, a few major scandals and people would flee from that browser like crazy. (I think almost every major browser dev here has been though such a crappy event and it ain't fun.) Now, I'm no USB expert, but isn't it possible for a USB device to provide a user level driver when being plugged in? If so then do that for the device (userlevel USB HID device driver?) Then provide a url to the webapp. The browser will/should ask the user if it's ok for webapp z to access the blah device, user clicks yes and off ya go. Doesn't sound that overly complicated to me (from a user standpoint). As a dev I know that an admin can (and should be able to) disable userlevel drivers (or ability to use them) etc. for some regular users in the OS. Likewise a school, library or public system or public service system might want to config the browser to not allow webapps to access hardware directly, in which case the browser would either turn up with a box saying No Devices Found or This Device is not allowed on this system or something.. If we allow webapps to tunnel straight through the Browser, the User, the OS, the Drivers, and access the Device directly you are
Re: [whatwg] Device Element
On 2010-12-28 03:16, Seth Brown wrote: I also believe that the working group should make the device element spec a high priority. If you don't google will probably implement their own version for chrome OS(it will be necessary in a browser based OS model). Thanks, Seth Brown Not really! Chrome OS will do Webapps the same way as Firefox, Safari, Opera, and IE will do it. What probably Chrome OS will do that the browsers don't do is Apps kinda like how there are Android Apps, and iPhone Apps. Webapps still work on Android and iPhone right? So it will hardly be any different on Chrome OS. An App will always have more control than a Webapp, and a driver will always have more control than a App, especially if the driver lives in a different privilege level. -- Roger Rescator Hågensen. Freelancer - http://www.EmSai.net/
Re: [whatwg] Device Element
First I'd like to point out that accessing hardware is already possible through ActiveX, and I think I've heard of some people doing it with XUL. I would like to standardize these hardware interactions. I don't think it should be done without strict security measures, but the ones mentioned in Roger H?gensen's reply would be sufficient to make this doable. Further security could be implemented by requiring the user to change a setting in the preferences pane, and then specifying a whitelist of devices and sites. Also it should be possible to allow an administrator to set policies to block the browser from accessing usb/RS232. Any software that you install has the ability to jump in and take control of RS232 devices. I see no more vulnerability if a user is required to explicitly grant permission to access a device than if a site prompts a user to install software that does the same thing. Either way the action wouldn't be taken until the user consents. At some point it is going to be necessary for the web apps to interact with hardware. The whole point of the chrome OS is portability. If it relies on installed apps for functionality then it isn't really portable. One of the main advantages of a system like that is that, for example, I can log into any supported device and now I'm logged into MY computer -- everything is the same as it was with the computer at my house, and I can start working without downloading who knows how many apps. I hate the concept of platform specific apps and I think that once HTML5 if fully implemented apps will start to disappear. The only real hurdle left is hardware interaction, but it will happen at some point. Why don't we make sure it gets done right? I'm going to start working on a concrete proposal. I've never written one before, so any advice would be appreciated -- should I just look through other completed specs and format it accordingly? -Seth
Re: [whatwg] Device Element
Any software that you install has the ability to jump in and take control of RS232 devices. That's news to me. What if my terminal, HTML and PDF reader don't have access to every device attached to my system, any more than my MTA?
[whatwg] Device Element
Sorry I meant potentially has that ability. Any software you install could do this without you knowing it. Sent from my iPhone On Dec 28, 2010, at 8:12 PM, Bjartur Thorlacius svartma...@gmail.com wrote: Any software that you install has the ability to jump in and take control of RS232 devices. That's news to me. What if my terminal, HTML and PDF reader don't have access to every device attached to my system, any more than my MTA?
[whatwg] Device Element
I saw on the spec page that you were considering limiting the device element to audio/visual streams. I thinks this would be a mistake. I'm currently working on integrating serial port connected hardware with a web application I'm developing. The only solution is for the user to install local adapter software. This defeats the purpose of using a web app over desktop software. In order for web applications to be gain traction over desktop software, they must be able to interface with usb/RS232. I believe the security trade off is worth it. I also believe that the working group should make the device element spec a high priority. If you don't google will probably implement their own version for chrome OS(it will be necessary in a browser based OS model). Thanks, Seth Brown
Re: [whatwg] Device Element
On Tue, Dec 28, 2010 at 3:16 AM, Seth Brown lear...@gmail.com wrote: I saw on the spec page that you were considering limiting the device element to audio/visual streams. I thinks this would be a mistake. I'm currently working on integrating serial port connected hardware with a web application I'm developing. The only solution is for the user to install local adapter software. This defeats the purpose of using a web app over desktop software. In order for web applications to be gain traction over desktop software, they must be able to interface with usb/RS232. I believe the security trade off is worth it. I also believe that the working group should make the device element spec a high priority. If you don't google will probably implement their own version for chrome OS(it will be necessary in a browser based OS model). Thanks, Seth Brown Agreed. Access to RS232 data from scripts will promote the use of web technologies in new applications for the embedded market. -- Diego
[whatwg] Device Element
Having a look over the draft, I notice a comment about the device element, RS232 is only included below to give an idea of where we could go with this. Should we instead just make this only useful for audiovisual streams? Unless there are compelling reasons, we probably should not be this generic. So far, the reasons aren't that compelling. I have a usage case that would benefit from other types of device access other than audiovisual: We currently run a news service that requires users log on to access our data and market reports (practically all major banking institutions in the world use us). I could envisage either a thumb-print reader allowing us to authenticate the user biometrically, or providing each user with a USB thumb stick that contains a unique identifier of some sort that when read using the device element could be used to authenticate them like a sort-of web-dongle. One of our big issues is plagiarism and password sharing amongst our clients. This type of access would allow us to lock down secure content without having to install applications on the client computers which as you can imagine, is a no-no when dealing with banks! Comments anyone?
Re: [whatwg] device element, streams and peer-to-peer connections
On May 31, 2010, at 03:01 , James Salsman wrote: On Sat, May 29, 2010 at 4:57 PM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: On Thu, May 27, 2010 at 9:26 PM, James Salsman jsals...@gmail.com wrote: Would it be appropriate to allow selection between reliable delivery involving delay and unreliable delivery with the shorter delay characteristics of UDP by allowing the user to select between the TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input type=file accept=audio as per http://www.w3.org/TR/device-upload and use UDP for synchronous or asynchronous device element I/O? I can see use cases for both methods -- a voice mail, server based application could use a simple form submit upload, but a live voice conferencing app would need real-time access more like in the Capture API that the W3C DAP group has published: http://www.w3.org/TR/capture-api/ It's hard for me to take http://www.w3.org/TR/capture-api/#formatdata seriously. There are no references to open codecs or codec parameters; the only audio codec specified is audio/x-wav, which is a Microsoft-defined union type (RIFF) with a huge number of different possible instance types, including only a few poor quality open vocoders and audio codecs by contemporary performance/bandwidth standards. Where is speex or ogg vorbis? Where are their quality and bit rate parameters? Why is http://www.w3.org/TR/capture-api/#future empty when most of the normative sections say, No exceptions? Where is the compatibility with existing file transfer standards? The security section doesn't contemplate permissions revocation. When a specification is fully complete, mature, and stable, we tend to release it. A rather shocking consequence of this policy is that drafts tend to be incomplete. What's even more outrageous is that sections explicitly marked as under development would be lacking in the contemplation of certain aspects department. The next thing you know it might turn out that Céline Dion karaoke sounds bad. Two things you might not know though: 1) the current draft does not look at streamed transmission, merely integration with file upload form controls and audio/video through reuse of the File API (this can be discussed of course, but see below) and 2) we're waiting on input we expect from our friends at Mozilla who have some ideas (but alas little time to write). One interesting suggestion they made was that capture should *not* include a direct way of streaming to a server. Rather, it should solely define how to wire a device to audio, video, img, etc. and there ought to be a separate document describe how to capture the output of an element and AV-stream it to a remote endpoint (or save it to disk). The reason for this is that there's no good reason why you wouldn't want to stream (or save) as AV things that originally aren't. There are plenty of use cases from capturing tutorials to the craziest collaborative film-making happening you can think of and including making a video effects editor inside the browser. Naturally, that's where protocol and format noodling would go. Personally, I think that that distinction makes sense. Note that going forward you'll probably want to look at http://dev.w3.org/2009/dap/camera/ rather than the TR snapshots. -- Robin Berjon - http://berjon.com/
Re: [whatwg] device element, streams and peer-to-peer connections
On Sun, May 30, 2010 at 6:01 PM, James Salsman jsals...@gmail.com wrote: It's hard for me to take http://www.w3.org/TR/capture-api/#formatdata seriously. There are no references to open codecs or codec parameters; the only audio codec specified is audio/x-wav, which is a Microsoft-defined union type (RIFF) with a huge number of different possible instance types, including only a few poor quality open vocoders and audio codecs by contemporary performance/bandwidth standards. Where is speex or ogg vorbis? Where are their quality and bit rate parameters? Why is http://www.w3.org/TR/capture-api/#future empty when most of the normative sections say, No exceptions? Where is the compatibility with existing file transfer standards? The security section doesn't contemplate permissions revocation. If audio were segmented into separate files as per http://www.w3.org/TR/capture-api/#captureaudiooptions how would that affect real-time performance on mobile devices? Are these files required to have sequence numbers? With phase vocoder time shifting, UDP delivery as per http://dev.w3.org/html5/html-device/#stream-api would be far superior in quality and intelligibility under packet loss or delay, assuming they went with an open audio codec (or, even better, allowed a choice of speex or ogg vorbis.) To be clear I'm not advocating for one particular capture API or codec; rather I'm advocating that capture and record not be tied to network transport, and separately that the p2p network transport be flexible, low-level, low-overhead and have a minimal attack surface (suitable for real-time game data as well as audio/video). Where is the discussion regarding phase vocoder time shifting and UDP delivery? I didn't see it in the stream-api section. Cheers, Mark
Re: [whatwg] device element, streams and peer-to-peer connections
On Mon, May 31, 2010 at 7:07 AM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: On Sun, May 30, 2010 at 6:01 PM, James Salsman jsals...@gmail.com wrote: ... With phase vocoder time shifting, UDP delivery as per http://dev.w3.org/html5/html-device/#stream-api would be far superior in quality and intelligibility under packet loss or delay, assuming they went with an open audio codec (or, even better, allowed a choice of speex or ogg vorbis.) ... Where is the discussion regarding phase vocoder time shifting and UDP delivery? I didn't see it in the stream-api section. Some vendors such as Vivox use phase vocoder time shifting to shorten audio segments to catch delayed streams back up to the wall clock. It works even better with TCP or RTP -- http://www.ietf.org/rfc/rfc3550.txt -- as with UDP, because the application layer programmer just has to re-invent the transmission control wheel with UDP anyway.
Re: [whatwg] device element, streams and peer-to-peer connections
On Sat, May 29, 2010 at 4:57 PM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: On Thu, May 27, 2010 at 9:26 PM, James Salsman jsals...@gmail.com wrote: Would it be appropriate to allow selection between reliable delivery involving delay and unreliable delivery with the shorter delay characteristics of UDP by allowing the user to select between the TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input type=file accept=audio as per http://www.w3.org/TR/device-upload and use UDP for synchronous or asynchronous device element I/O? I can see use cases for both methods -- a voice mail, server based application could use a simple form submit upload, but a live voice conferencing app would need real-time access more like in the Capture API that the W3C DAP group has published: http://www.w3.org/TR/capture-api/ It's hard for me to take http://www.w3.org/TR/capture-api/#formatdata seriously. There are no references to open codecs or codec parameters; the only audio codec specified is audio/x-wav, which is a Microsoft-defined union type (RIFF) with a huge number of different possible instance types, including only a few poor quality open vocoders and audio codecs by contemporary performance/bandwidth standards. Where is speex or ogg vorbis? Where are their quality and bit rate parameters? Why is http://www.w3.org/TR/capture-api/#future empty when most of the normative sections say, No exceptions? Where is the compatibility with existing file transfer standards? The security section doesn't contemplate permissions revocation. If audio were segmented into separate files as per http://www.w3.org/TR/capture-api/#captureaudiooptions how would that affect real-time performance on mobile devices? Are these files required to have sequence numbers? With phase vocoder time shifting, UDP delivery as per http://dev.w3.org/html5/html-device/#stream-api would be far superior in quality and intelligibility under packet loss or delay, assuming they went with an open audio codec (or, even better, allowed a choice of speex or ogg vorbis.) Regards, James Salsman
Re: [whatwg] device element, streams and peer-to-peer connections
On Thu, May 27, 2010 at 9:26 PM, James Salsman jsals...@gmail.com wrote: Would it be appropriate to allow selection between reliable delivery involving delay and unreliable delivery with the shorter delay characteristics of UDP by allowing the user to select between the TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input type=file accept=audio as per http://www.w3.org/TR/device-upload and use UDP for synchronous or asynchronous device element I/O? I can see use cases for both methods -- a voice mail, server based application could use a simple form submit upload, but a live voice conferencing app would need real-time access more like in the Capture API that the W3C DAP group has published: http://www.w3.org/TR/capture-api/ . As they've laid it out, capture of audio/video is decoupled from the network transmission/streaming of the captured data, which makes sense. The media file data captured could then be sliced into blobs and bounced off a server via WebSocket or sent to peers via the peer to peer API. Again here it would make sense to me to pattern the p2p API more closely to WebSocket (i.e. send packets of bytes) than a higher level approach that tries to multiplex streams of data. - Mark
Re: [whatwg] device element, streams and peer-to-peer connections
-Original Message- From: whatwg-boun...@lists.whatwg.org [mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of Anne van Kesteren Sent: den 21 maj 2010 10:30 To: wha...@whatwg.org; Nicklas Sandgren Subject: Re: [whatwg] device element, streams and peer-to-peer connections On Fri, 21 May 2010 10:20:00 +0200, Nicklas Sandgren nicklas.sandg...@ericsson.com wrote: As mentioned in the draft, the peer-to-peer API must rely on underlying protocols/mechanisms to establish the connections and to transport the streams. What are the thoughts regarding these protocols, and has there been any discussion around this topic? Last I checked the hope is that the protocol problem will go away. So far it seems that is unlikely. :-) An alternative approach could be to define APIs for managing streams only, and leave session set up as well as additional functionality (file, text, image share) to the application using the means already available such as XMLHttpRequest and WebSocket. The session set up would in this scenario not rely on a third party server, but rather be handled by the server that serves the current web application. This would remove the need for agreeing on formats for client and server configuration strings or protocols to talk to third-party servers. You could also debate how often peer-to-peer media streams will actually work. Aren't FWs and NATs going to give problems in many cases? Maybe it would be better to design for a situation where the media always go via a server. Additional benefits are that WS could be used for media transport, and that the media could be transcoded if the codec capabilities of the clients do not match. I'm not really sure how this is an alternative approach. It would just be leaving peer-to-messaging out... Streaming video via WebSocket is something we definitely want to enable in due course, irrespective of whether peer-to-peer messaging comes to fruition. What I meant was that if you just specify a lower level API used only for sending and receiving media streams, some of the that protocol problem would go away. As I see it the only functionality in the Peer-to-peer API that it is not currently possible to implement using other means (won't be p2p but the end result will more or less be the same) is the media stream part. Hence, in my opinion, that would be the most interesting feature to enable. --Nicklas
Re: [whatwg] device element, streams and peer-to-peer connections
On Fri, May 21, 2010 at 1:29 AM, Anne van Kesteren ann...@opera.com wrote: On Fri, 21 May 2010 10:20:00 +0200, Nicklas Sandgren nicklas.sandg...@ericsson.com wrote: As mentioned in the draft, the peer-to-peer API must rely on underlying protocols/mechanisms to establish the connections and to transport the streams. What are the thoughts regarding these protocols, and has there been any discussion around this topic? Last I checked the hope is that the protocol problem will go away. So far it seems that is unlikely. :-) There has been some limited discussion about the peer-to-peer section as it relates to real-time peer to peer gaming applications: http://old.nabble.com/Real-time-networking-in-web-applications-and-games-td27994693.html And references a proposed lower level (UDP-based) WebSocket-like protocol: http://github.com/nardo/torque_sockets/raw/master/TorqueSocket_API.txt The approach here was to punt as much of the flow control/retransmission policy to the application level, whether you're streaming video streams, game data or whatever. You could also debate how often peer-to-peer media streams will actually work. Aren't FWs and NATs going to give problems in many cases? Maybe it would be better to design for a situation where the media always go via a server. Additional benefits are that WS could be used for media transport, and that the media could be transcoded if the codec capabilities of the clients do not match. I'm not really sure how this is an alternative approach. It would just be leaving peer-to-messaging out... Streaming video via WebSocket is something we definitely want to enable in due course, irrespective of whether peer-to-peer messaging comes to fruition. To answer the question of problem in p2p regarding FWs and NATs, the libjingle folks report that 92% of participants are able to connect directly: http://code.google.com/apis/talk/libjingle/important_concepts.html#connections with the remainder using simple message relay servers. In practice this represents a hugely significant bandwidth reduction for application hosts. Given the level of possible application sophistication promise in the next gen web, leaving out such a huge efficiency win would be a shame. An approach that allows data output from devices to be adjusted to a target bit rate and collected in meaningful chunks, but not directly connected to the transport mechanism could be a viable path. A device stream could be transported either via WebSocket or a p2p networking layer with equal ease, leaving the message marshaling up to the web app. Any feedback on the specific p2p packet layer protocol is always welcome. Regards, Mark
Re: [whatwg] device element, streams and peer-to-peer connections
On Thu, May 27, 2010 at 3:34 PM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: There has been some limited discussion about the peer-to-peer section as it relates to real-time peer to peer gaming applications: http://old.nabble.com/Real-time-networking-in-web-applications-and-games-td27994693.html And references a proposed lower level (UDP-based) WebSocket-like protocol: http://github.com/nardo/torque_sockets/raw/master/TorqueSocket_API.txt The approach here was to punt as much of the flow control/retransmission policy to the application level, whether you're streaming video streams, game data or whatever. Why is relying on TCP for reliable delivery inferior to asking applications to re-implement reliable transmission?
Re: [whatwg] device element, streams and peer-to-peer connections
On Thu, May 27, 2010 at 6:22 PM, James Salsman jsals...@gmail.com wrote: Why is relying on TCP for reliable delivery inferior to asking applications to re-implement reliable transmission? In real-time networked applications the retransmission delay imposed by TCP can cause unnecessary and noticeable hitches in the experience; hence many real-time games use custom network stacks built atop UDP -- for a more in-depth read you can check out (somewhat dated at this point but largely relevant) : http://www710.univ-lyon1.fr/~jciehl/Public/educ/GAMA/2008/tribes_networking_model.pdf . This is not to suggest TCP is somehow inferior, just that for certain applications it's not the right tool. The next gen web has all the pieces in at least prototype form to support full, high-performance apps, but without low-level networking support (real-time, p2p) whole classes of applications will be excluded.
Re: [whatwg] device element, streams and peer-to-peer connections
On Thu, May 27, 2010 at 7:59 PM, Mark Frohnmayer mark.frohnma...@gmail.com wrote: On Thu, May 27, 2010 at 6:22 PM, James Salsman jsals...@gmail.com wrote: Why is relying on TCP for reliable delivery inferior to asking applications to re-implement reliable transmission? In real-time networked applications the retransmission delay imposed by TCP can cause unnecessary and noticeable hitches in the experience; hence many real-time games use custom network stacks built atop UDP Would it be appropriate to allow selection between reliable delivery involving delay and unreliable delivery with the shorter delay characteristics of UDP by allowing the user to select between the TCP-based asynchronous HTTP/HTTPS multipart/form-encoded POST of input type=file accept=audio as per http://www.w3.org/TR/device-upload and use UDP for synchronous or asynchronous device element I/O? Best regards, James Salsman
[whatwg] device element, streams and peer-to-peer connections
Looking at the latest draft, section 4.11.6 contains a proposal for the device element as discussed quite extensively on the W3C DAP list. In addition there is a Stream API and a sub-section on peer-to-peer connections. As mentioned in the draft, the peer-to-peer API must rely on underlying protocols/mechanisms to establish the connections and to transport the streams. What are the thoughts regarding these protocols, and has there been any discussion around this topic? An alternative approach could be to define APIs for managing streams only, and leave session set up as well as additional functionality (file, text, image share) to the application using the means already available such as XMLHttpRequest and WebSocket. The session set up would in this scenario not rely on a third party server, but rather be handled by the server that serves the current web application. This would remove the need for agreeing on formats for client and server configuration strings or protocols to talk to third-party servers. You could also debate how often peer-to-peer media streams will actually work. Aren't FWs and NATs going to give problems in many cases? Maybe it would be better to design for a situation where the media always go via a server. Additional benefits are that WS could be used for media transport, and that the media could be transcoded if the codec capabilities of the clients do not match. --Nicklas Sandgren
Re: [whatwg] device element, streams and peer-to-peer connections
On Fri, 21 May 2010 10:20:00 +0200, Nicklas Sandgren nicklas.sandg...@ericsson.com wrote: As mentioned in the draft, the peer-to-peer API must rely on underlying protocols/mechanisms to establish the connections and to transport the streams. What are the thoughts regarding these protocols, and has there been any discussion around this topic? Last I checked the hope is that the protocol problem will go away. So far it seems that is unlikely. :-) An alternative approach could be to define APIs for managing streams only, and leave session set up as well as additional functionality (file, text, image share) to the application using the means already available such as XMLHttpRequest and WebSocket. The session set up would in this scenario not rely on a third party server, but rather be handled by the server that serves the current web application. This would remove the need for agreeing on formats for client and server configuration strings or protocols to talk to third-party servers. You could also debate how often peer-to-peer media streams will actually work. Aren't FWs and NATs going to give problems in many cases? Maybe it would be better to design for a situation where the media always go via a server. Additional benefits are that WS could be used for media transport, and that the media could be transcoded if the codec capabilities of the clients do not match. I'm not really sure how this is an alternative approach. It would just be leaving peer-to-messaging out... Streaming video via WebSocket is something we definitely want to enable in due course, irrespective of whether peer-to-peer messaging comes to fruition. -- Anne van Kesteren http://annevankesteren.nl/