[whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Chang Shu]

Hi, all,

Based on the discussion in Blink-dev mail group,
https://groups.google.com/a/chromium.org/forum/#!searchin/blink-dev/stringview/blink-dev/ylgiNY_ZSV0/vne3e0aplUUJ,
particularly from Adam Bath's suggestion, I drafted a proposal for
binary encoding on Typed Arrays by enhancing window.btoa and atob
APIs.

The text of the proposal is pasted as below. I appreciate your kind review.

Chang

---
BinaryEncoding

Proposed Binary Encoding Web API for Typed Arrays

Editors
Chang Shu (Samsung Electronics)

Abstract

This specification enhances the existing window.btoa and window.atob
Web APIs to encode directly from Typed Arrays to Base64 strings and
vice versa.

Use Case Description

A webapp sends data from Typed Array to NPAPI plugin and also reads
data back. Since the only possible data type supported by NPAPI to
send data buffer is text string, an efficient way of converting Typed
Array to Base64 string is required.

Current Limitations

The only existing Web API involving binary data encoding are
window.btoa and window.atob. However, they require the raw data being
contained in a 'binary' string format while in many cases the raw data
is stored in Typed Array.

Current Usage and Workarounds

Currently, converting Typed Array to 'binary' string can be done in JS
functions. But performance is not acceptable for large data buffers.

Benefits

The new enhancement of btoa and atob converts the raw data in Typed
Arrays to text string and from text string to Typed Arrays directly.
The performance is the best since all operations are done in native
code.

Requests for this Feature

I would like this feature to be implemented and shipped.

Proposed Solutions

My Solution

The approach is to enhance existing btoa and atob to take Typed Array
parameters while keeping backward compatibility.

window.btoa

Summary

Creates a base-64 encoded ASCII string from either a string of
binary data or a Typed Array.

Syntax

var encodedData = window.btoa(dataToEncode);

Note that there is no syntax change in window.btoa API.

Example

var encodedData = window.btoa(hello); //encode a string. Consider
the string as 'binary'

var arr = new Int32Array(3);
arr[0] = 1;
arr[1] = 2;
arr[2] = 3;
var encodedData = window.btoa(arr); //encode integer data into a base-64 string

window.atob

Summary

Decodes a base-64 encoded ASCII string into a string of binary data
and a Typed Array if parameter provided.

Syntax

var decodedArr = new Int32Array();
var decodedData = window.atob(encodedData, [Optinoal] decodedArr);

Note that the 2nd parameter is optional which keeps the backward compatibility.

Example

var arr = new Int32Array(3);
arr[0] = 1;
arr[1] = 2;
arr[2] = 3;
var encodedData = window.btoa(arr); //encode integer data into a base-64 string

var newarr = new Int32Array();
window.atob(encodedData, newarr); //decode base-64 string back to integer array
//newarr[0] should be 1, newarr[1] should be 2 and newarr[2] should be 3.

[whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Chang Shu]

Hi, all,

Based on the discussion in Blink-dev mail group,
https://groups.google.com/a/chromium.org/forum/#!searchin/blink-dev/stringview/blink-dev/ylgiNY_ZSV0/vne3e0aplUUJ,
particularly from Adam Bath's suggestion, I drafted a proposal for
binary encoding on Typed Arrays by enhancing window.btoa and atob
APIs.

The text of the proposal is pasted as below. I appreciate your kind review.

Chang

---
BinaryEncoding

Proposed Binary Encoding Web API for Typed Arrays

Editors
Chang Shu (Samsung Electronics)

Abstract

This specification enhances the existing window.btoa and window.atob
Web APIs to encode directly from Typed Arrays to Base64 strings and
vice versa.

Use Case Description

A webapp sends data from Typed Array to NPAPI plugin and also reads
data back. Since the only possible data type supported by NPAPI to
send data buffer is text string, an efficient way of converting Typed
Array to Base64 string is required.

Current Limitations

The only existing Web API involving binary data encoding are
window.btoa and window.atob. However, they require the raw data being
contained in a 'binary' string format while in many cases the raw data
is stored in Typed Array.

Current Usage and Workarounds

Currently, converting Typed Array to 'binary' string can be done in JS
functions. But performance is not acceptable for large data buffers.

Benefits

The new enhancement of btoa and atob converts the raw data in Typed
Arrays to text string and from text string to Typed Arrays directly.
The performance is the best since all operations are done in native
code.

Requests for this Feature

I would like this feature to be implemented and shipped.

Proposed Solutions

My Solution

The approach is to enhance existing btoa and atob to take Typed Array
parameters while keeping backward compatibility.

window.btoa

Summary

Creates a base-64 encoded ASCII string from either a string of
binary data or a Typed Array.

Syntax

var encodedData = window.btoa(dataToEncode);

Note that there is no syntax change in window.btoa API.

Example

var encodedData = window.btoa(hello); //encode a string. Consider
the string as 'binary'

var arr = new Int32Array(3);
arr[0] = 1;
arr[1] = 2;
arr[2] = 3;
var encodedData = window.btoa(arr); //encode integer data into a base-64 string

window.atob

Summary

Decodes a base-64 encoded ASCII string into a string of binary data
and a Typed Array if parameter provided.

Syntax

var decodedArr = new Int32Array();
var decodedData = window.atob(encodedData, [Optinoal] decodedArr);

Note that the 2nd parameter is optional which keeps the backward compatibility.

Example

var arr = new Int32Array(3);
arr[0] = 1;
arr[1] = 2;
arr[2] = 3;
var encodedData = window.btoa(arr); //encode integer data into a base-64 string

var newarr = new Int32Array();
window.atob(encodedData, newarr); //decode base-64 string back to integer array
//newarr[0] should be 1, newarr[1] should be 2 and newarr[2] should be 3.

Re: [whatwg] Forms: input type=file and directory tree picking

[Jonathan Watt]

On 02/08/2013 23:39, Glenn Maynard wrote:

On Fri, Aug 2, 2013 at 11:15 AM, Jonathan Watt jw...@jwatt.org wrote:


In my prototype implementation it took around 30 seconds to build the
FileList for a directory of 200,000 files with a top end SSD; so depending
on what the page is doing, directory picking could take some time.



A static list isn't appropriate for recursively exposing a large
directory.  I hope that won't be implemented, since that's the sort of
halfway-feature--not quite good enough, but it sort of works--that can
delay a good API indefinitely.  An interface to allow scripts to navigate
the tree like a filesystem should be used, to avoid having to do a full
recursion.


I see this more as a means of getting directory picking working in the existing 
content out there that's using file pickers without requiring that content to be 
updated (which is not going to happen in the vast majority of cases). I would 
hope that it would not _prevent_ a filesystem-like API from being implemented, 
although I agree it would inevitably reduce the pressure for such an API. That 
said, such an API that everyone agrees on doesn't seem to be materializing any 
time soon.



For example, a photo browser probably only wants to read data on demand, as
the user navigates.  Also, doing it synchronously means that if the user
adds another photo, he'd have to reopen the directory (and wait for the
long recursion) all over again for it to be seen by the app.

A previous discussion (on drag and drop, but the issues are the same) is
here:

http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2011-November/033814.html


Thanks for the link. I've read through that thread.

-Jonathan


That centered around FS-API, which is probably not the direction things are
going, but whichever API things land on for filesystem access should
probably be used for this--or vice-versa, if this comes first.  I suspect
they're actually the same thing, since a file picker (along with drag and
drop) is the likely way to expose a filesystem-of-real-files API.

Re: [whatwg] Forms: input type=file and directory tree picking

[Jonathan Watt]

On 05/08/2013 00:54, Glenn Maynard wrote:

On Sun, Aug 4, 2013 at 2:47 AM, Jonas Sicking jo...@sicking.cc wrote:


We can't do what you are suggesting for a plain input type=file
multiple since there's already a defined API for that, and that API
only exposes a .files property in the DOM.



Sure we can; we can always add to that API, such as adding a
getFileSystem() method.  A different @type may be better anyway, though.


Currently my own view is that exposing a filesystem type API must be opt-in (as 
in new code has to be written to use it; old code isn't going to just work). 
As such a new attribute [value] such as multiple=fs could be required to make 
use of it, and could change the behavior of the input so that it doesn't build 
up a FileList, but instead sets .files to an object representing the picked 
directory if a directory is picked.


-Jonathan


  But we could certainly add some way to enable creating an input

which exposes files and directories in the DOM, rather than just
files. Doing that will depend on coming up with a filesystem proposal
which all parties actually agree on implementing, so far we don't have
such a proposal.



Unless we think it won't ever happen, it'd be better to keep working
towards that than to rush things and implement greedy recursion.

  It also requires an actual proposal for what such an input would

look like. I.e. would it be an input type=file directory? Or input
type=file multiple with some sort of API call saying that flattening
directories into files aren't needed? Or input
type=filesanddirectories?



It's probably not worth worrying about this part too much until we have a
filesystem API for it to enable.  Any of these seem fine (though I'd lean
away from an API call), or maybe input type=file multiple=fs, which would
cause it to fall back on the current (files only, non-recursive FileList)
behavior on browsers that don't support it.

(I don't think flattening directories into files is something that should
ever happen in the first place, but if it does we'd definitely need to make
sure it doesn't happen in this mode.)

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Simon Pieters]

On Mon, 05 Aug 2013 16:10:50 +0200, Chang Shu csh...@gmail.com wrote:


window.btoa

Summary

Creates a base-64 encoded ASCII string from either a string of
binary data or a Typed Array.

Syntax

var encodedData = window.btoa(dataToEncode);

Note that there is no syntax change in window.btoa API.

Example

var encodedData = window.btoa(hello); //encode a string. Consider
the string as 'binary'

var arr = new Int32Array(3);
arr[0] = 1;
arr[1] = 2;
arr[2] = 3;
var encodedData = window.btoa(arr); //encode integer data into a base-64  
string


window.atob

Summary

Decodes a base-64 encoded ASCII string into a string of binary data
and a Typed Array if parameter provided.

Syntax

var decodedArr = new Int32Array();
var decodedData = window.atob(encodedData, [Optinoal] decodedArr);

Note that the 2nd parameter is optional which keeps the backward  
compatibility.


Example

var arr = new Int32Array(3);
arr[0] = 1;
arr[1] = 2;
arr[2] = 3;
var encodedData = window.btoa(arr); //encode integer data into a base-64  
string


var newarr = new Int32Array();
window.atob(encodedData, newarr); //decode base-64 string back to  
integer array

//newarr[0] should be 1, newarr[1] should be 2 and newarr[2] should be 3.


Is there a reason to support an arbitrary typed array for atob rather than  
returning a new typed array?


e.g.

var newarr = atob(encodedData, {typedarray:true});

(I'm not sure which view is most appropriate to return.)

--
Simon Pieters
Opera Software

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Tab Atkins Jr.]

On Mon, Aug 5, 2013 at 12:41 PM, Simon Pieters sim...@opera.com wrote:
 var newarr = new Int32Array();
 window.atob(encodedData, newarr); //decode base-64 string back to integer
 array
 //newarr[0] should be 1, newarr[1] should be 2 and newarr[2] should be 3.

 Is there a reason to support an arbitrary typed array for atob rather than
 returning a new typed array?

 e.g.

 var newarr = atob(encodedData, {typedarray:true});

 (I'm not sure which view is most appropriate to return.)

Decoding into a given typed array means you can reuse the same
(potentially large) buffer multiple times, rather than creating new
ones with every call.  Avoiding the GC churn can be significant.  It
also means you can decode several strings into the same typed array at
different offsets.

~TJ

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Chang Shu]

I see your point now, Simon. Technically both approaches should work.
As you said, yours has the limitation that the implementation does not
know which view to return unless you provide an enum type of parameter
instead of boolean to atob. And mine has the performance issue. How
about we don't return the 'binary' string in case the 2nd parameter is
provided in my case?

Thanks


On Mon, Aug 5, 2013 at 4:22 PM, Simon Pieters sim...@opera.com wrote:
 On Mon, Aug 5, 2013 at 3:41 PM, Simon Pieters sim...@opera.com wrote:

 Is there a reason to support an arbitrary typed array for atob rather
 than
 returning a new typed array?

 e.g.

 var newarr = atob(encodedData, {typedarray:true});

 (I'm not sure which view is most appropriate to return.)


 On Mon, 05 Aug 2013 21:50:39 +0200, Chang Shu csh...@gmail.com wrote:

 Thanks for the comments, Simon. The reason I put the return result as
 a second (optional) parameter of atob is for backward compatibility.
 User can still call:
 var rtn = atob(Base64EncodedData);
 and rtn would be a 'binary' string.
 If user calls:
 var rtn = atob(Base64EncodedData, arrayBufferView);
 arrayView would be filled with data and rtn is also a parsed 'binary'
 string. I think the native implementation can detect what type of
 ArrayBufferView arrayBufferView is.


 I think you misunderstood my question.

 I understand that

 var rtn = atob(Base64EncodedData);

 still needs to work for backwards compabitility. But that doesn't explain
 why


 var rtn = atob(Base64EncodedData, arrayBufferView);

 is better than


 var newarr = atob(encodedData, {typedarray:true});

 Your suggestion to both return a 'binary' string and to fill in the typed
 array provided in the second argument seems like it would have bad
 performance compared to only filling in or returning a typed array.


 --
 Simon Pieters
 Opera Software

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Chang Shu]

That's a very good point, TJ. Thanks.

Chang

On Mon, Aug 5, 2013 at 4:30 PM, Tab Atkins Jr. jackalm...@gmail.com wrote:
 On Mon, Aug 5, 2013 at 12:41 PM, Simon Pieters sim...@opera.com wrote:
 var newarr = new Int32Array();
 window.atob(encodedData, newarr); //decode base-64 string back to integer
 array
 //newarr[0] should be 1, newarr[1] should be 2 and newarr[2] should be 3.

 Is there a reason to support an arbitrary typed array for atob rather than
 returning a new typed array?

 e.g.

 var newarr = atob(encodedData, {typedarray:true});

 (I'm not sure which view is most appropriate to return.)

 Decoding into a given typed array means you can reuse the same
 (potentially large) buffer multiple times, rather than creating new
 ones with every call.  Avoiding the GC churn can be significant.  It
 also means you can decode several strings into the same typed array at
 different offsets.

 ~TJ

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Simon Pieters]

On Mon, 05 Aug 2013 22:39:22 +0200, Chang Shu csh...@gmail.com wrote:


I see your point now, Simon. Technically both approaches should work.
As you said, yours has the limitation that the implementation does not
know which view to return unless you provide an enum type of parameter
instead of boolean to atob. And mine has the performance issue. How
about we don't return the 'binary' string in case the 2nd parameter is
provided in my case?


That works for me.

--
Simon Pieters
Opera Software

Re: [whatwg] web messaging - postMessage

[Ian Hickson]

On Sun, 27 Jan 2013, Jack (Zhan, Hua Ping) wrote:
 
 The postMessage design outlined in the W3C document edited by Ian 
 Hickson is not good! The design of the cross document messaging by Ian 
 Hickson (Google, Inc.) is very bad. Even the last version is not good 
 either.

:-(


 The design can be sketched here as follows.
 
 The sender:
 var o = document.getElementsByTagName('iframe')[0];
 o.contentWindow.postMessage('Hello world', 'http://b.example.org/');
 
 The receiver:
 window.addEventListener('message', receiver, false);
 function receiver(e) {
   if (e.origin == 'http://example.com') {
 if (e.data == 'Hello world') {
   e.source.postMessage('Hello', e.origin);
 } else {
   alert(e.data);
 }
   }
 }
 
 This design was messed up by pulling origin (a word that some people
 put too much attention more than should).

Can you elaborate on what you mean by pulling in this sentence?


 Even worse, it requires o.contentWindow, this is really no 
 professional sense. Because of this design, if I open two tabs with the 
 same url http://www.google.com/ they are not able to communicate.

If you need to communicate between two otherwise unrelated tabs, there are 
basically two solutions: a shared worker, or a broadcast mechanism. The 
latter isn't currently specced, but discussion on how to do it is here:

   https://www.w3.org/Bugs/Public/show_bug.cgi?id=22628

You can also currently fake this using the 'onstorage' event.


 My better proposal
 
 the sender:
 window.postMessage(messageObject,targetDomain optional,windowIDs optional);
 
 Either targetDomain or windowIDs should present.
 I propose to use ID rather than name (though window can have a name),
 since window.name is not required to be unique within the browser.

This is basically a broadcast mechanism, right.


One things that would be useful in making progress on this would be more 
use cases, in particular use cases where it's important to be able to 
respond after a broadcast.

There's basically only one use case in the bug above, namely informing 
other windows of a state change (e.g. user logged in). This can currently 
be done using onstorage and that could arguably in fact be a sufficient 
solution (since for a change notification, there's no need to respond, 
and it's typically stored state that has changed).

Cheers,
-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] XML data islands related question

[Ian Hickson]

On Thu, 7 Feb 2013, Neerja Anand wrote:

 Hello - it appears that XML data islands (originally created by 
 Microsoft) have been discontinued in the newer browsers. We wanted to 
 request they could be supported as an addon/plugin into the browser if 
 not incorporated into the browser by default. We understand that this 
 was a Microsoft only feature but they are extremely helpful to our 
 business and their discontinuation would highly affect our business. Our 
 request is to make them as an HTML5 standard, so all browsers would then 
 have to support this feature. This would make the life of us developers 
 much easier. We are not asking for Microsoft's version of XML data 
 islands specifically, but would want this feature as a standard in some 
 version. We noticed there is something new called Data Block which may 
 be similar to XML data islands, but couldn't find much information on it 
 to proceed. If you know of Data Blocks being an alternative to XML data 
 islands, then if you could provide some documentation or point us to the 
 location where we can get more information (with examples on how to use 
 them) that would be very helpful as well.

Could you elaborate on the use case? Why do you use them / want to use 
them? What are they for?


On Thu, 7 Feb 2013, Mat Carey wrote:
 
 I'm not particularly active on WHATWG but the general approach seems to 
 be that if you have a good idea you should go and create it, show that 
 it works and then bring it to WHATWG for a wider adoption.

That certainly is a solid way of approaching things.


On Thu, 7 Feb 2013, Randy wrote:

 As far as I'm aware, Data Blocks should be or is part of HTML5. See 
 https://developer.mozilla.org/en/docs/Using_XML_Data_Islands_in_Mozilla 
 for more info.

On Thu, 7 Feb 2013, Jukka K. Korpela wrote:

 [...] browsers do not seem to support using a src attribute then; the 
 non-script data must be inline, as element content, not fetched from an 
 external source. I suppose the usual suspects (security considerations) 
 are to be blaimed. This however means that the technique is not 
 comparable to xml, which allowed external references.

Right.


 It might be useful to mention explicitly in the spec that browsers do 
 not generally allow non-script to be specified via the src attribute. 

The spec seems pretty clear about this:

# When used to include data blocks (as opposed to scripts), the data must 
# be embedded inline, the format of the data must be given using the type 
# attribute, the src attribute must not be specified, and the contents of 
# the script element must conform to the requirements defined for the 
# format used.

(This paragraph of script section.)


 And perhaps the spec should say that this is implementation-dependent 
 and recommend that such references should be allowed, with due security 
 considerations.

Implementation-dependent behaviour is, on the whole, just a source of 
confusion for authors, so I'd rather not do that.

You can just use XHR, instead.


 It's a bit odd that if you wish to set up a standalone application 
 running in a browser (often called HTML5 application, without implying 
 any particular version of HTML5), you can include e.g. scripts and 
 images in separate files but not plain text or XML data

Why can't you put plain text or XML data in other files? So long as 
everything is same origin, you can read anything you want via XHR.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] NoDatabase databases

[Ian Hickson]

On Thu, 2 May 2013, Brett Zamir wrote:
 
 I wanted to propose (if work has not already been done in this area) 
 creating an HTTP extension to allow querying for retrieval and updating 
 of portions of HTML (or XML) documents where the server is so capable 
 and enabled, obviating the need for a separate database (or more 
 accurately, bringing the database to the web server layer).

 1) Allowing one-off queries to be made by (privileged) user agents. This 
 avoids the need for websites willing to share their data to create their 
 own database APIs and overhead while allowing both the client and server 
 the opportunity to avoid delivering content which is not of interest to 
 the user. Possible query languages might include CSS selectors, XPath, 
 XQuery, or JavaScript.
 
 2) Allowing third-party websites the ability to make such queries of 
 other sites as in #1 but requiring user permission. I seem to recall 
 seeing some discussions apparently reviving the possibility for 
 JavaScript APIs to make cross-domain requests with user permission 
 regardless of the target site giving permission.

I don't really understand these use cases. When would you want to do this?

A use case really should be drawn all the way back to a user need, if 
possible. What user-facing application would you envisage where this 
feature would be needed to achieve a good experience?


 3) The ability for user agents to allow the user to provide intelligent 
 defaults for navigating a subset of potentially large data documents, 
 potentially with the assistance of website mark-up, but without the need 
 for website scripting. This could reduce development time and costs, 
 while ensuring that powerful capabilities were enabled for users by 
 default on all websites (at least those that opted in by a simple 
 server-side configuration option). It could also avoid unnecessary 
 demands on the server and wait-time for the client (benefiting energy 
 usage, access in developing countries, wait-times anywhere for large 
 documents, etc.), while conversely facilitating exposure by sites of 
 large data-sets for users wishing to download a large data set. 
 Web-based IDEs, moreover, could similarly allow querying and editing of 
 these documents without needing to load and display the full data set 
 during editing. Some concrete examples include:
 
 a) Allowing ordered or unordered lists or definition/dialogue lists 
 or any hierarchical markup to be navigated upon user demand. The client 
 and server might, for example, negotiate the number of list items from a 
 list to be initially loaded and shown such that the entire list would 
 not be displayed or loaded but instead would load say only the first and 
 last 5 items in the list and give the user a chance to manually load the 
 rest if they were interested in viewing all of that data. Hierarchical 
 lists, moreover, could allow Ajax-like drill-down capabilities (or if 
 the user so configured their user agent, to automatically expand to a 
 certain depth), all without the author needing to provide any scripting, 
 letting them focus on content. Even non-list markup, like paragraphs, 
 could be drilled into, as well as providing ellipses when the child 
 content was determined to be above a given memory size or if the element 
 was conventionally used to provide larger amounts of data (e.g., a 
 textarea). (Form submission would probably need to be disabled though 
 until all child content was loaded, and again, in order to avoid usage 
 against the site's intended design, such navigation might require 
 opt-in.)

For this to be a good feature, the page would have to be very large (many 
megabytes, on modern connections), or, the user's connection would have to 
be really constrained (low bandwidth, though for sanity the latency would 
still have to be low).

What such cases exist, where scripting wouldn't be desireable for other 
reasons?


 b) Tables would merit special treatment as a hierarchical type as 
 one may typically wish to ensure that all cells in a given row were 
 shown by default (though even here, ellipses could be added when the 
 data size was determined to be large), with pagination being the 
 well-used norm of table-based widgets. Having markup specified on column 
 headers (if not full-blown schemas) to indicate data types would be 
 useful in this regard (markup on the top level of a list might similarly 
 be useful); if the user agent were, for example, made aware of the fact 
 that a table column consisted exclusively of dates, it would provide a 
 search option to allow the user to display records between a given date 
 range (as well as better handling sorting).
 
 Rows could, moreover be auto-numbered by the agent with an option to 
 choose a range of numbers (similarly ranges could be provided for other 
 elements, like paragraph or list item numbering, etc.). The shift to the 
 user agent might also encourage the ability to reorder or remove 
 

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Kenneth Russell]

On Mon, Aug 5, 2013 at 2:04 PM, Simon Pieters sim...@opera.com wrote:
 On Mon, 05 Aug 2013 22:39:22 +0200, Chang Shu csh...@gmail.com wrote:

 I see your point now, Simon. Technically both approaches should work.
 As you said, yours has the limitation that the implementation does not
 know which view to return unless you provide an enum type of parameter
 instead of boolean to atob. And mine has the performance issue. How
 about we don't return the 'binary' string in case the 2nd parameter is
 provided in my case?


 That works for me.

Chang, in your proposal for modifying atob, how does the developer
know how many characters were written into the outgoing
ArrayBufferView?

What happens if the ArrayBufferView argument isn't large enough to
hold the decoded string?

During the decoding process, is the type of the ArrayBufferView
significant? In your example, what would happen if an Int16Array were
passed instead of an Int32Array?

The Encoding spec at http://encoding.spec.whatwg.org/ seems to have
handled issues like these. Perhaps a better route would be to fold
this functionality into that spec.

-Ken

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Chang Shu]

Hi, Kenneth,

I think the atob implementation is able to enlarge the buffer size
from c++ side if necessary. And during the decoding process, I thought
the algorithm first decodes base64 string into a binary string and
casts/copies it into the desired type based on the input
ArrayBufferView. So it is the user who decides the type of the output
not the encoded string. I think it's the same thing for the current
atob function. The encoded string has no information about what the
type of the output would be. Do you think the above is correct?
There have been some email exchanges about the different approaches of
binary encoding. I was told that the binary encoding part was taken
out of the encoding spec on purpose and the bota/atob approach was
preferred. But people are also talking about bringing it back to the
encoding spec. My bottom line is to have something working so I can
use it in my project. btoa/atob seems a natural enhancement and
low-hanging fruit to me.

Thanks,
Chang

 Chang, in your proposal for modifying atob, how does the developer
 know how many characters were written into the outgoing
 ArrayBufferView?

 What happens if the ArrayBufferView argument isn't large enough to
 hold the decoded string?

 During the decoding process, is the type of the ArrayBufferView
 significant? In your example, what would happen if an Int16Array were
 passed instead of an Int32Array?

 The Encoding spec at http://encoding.spec.whatwg.org/ seems to have
 handled issues like these. Perhaps a better route would be to fold
 this functionality into that spec.

 -Ken

Re: [whatwg] BinaryEncoding for Typed Arrays using window.btoa and window.atob

[Boris Zbarsky]

On 8/5/13 9:15 PM, Chang Shu wrote:

I think the atob implementation is able to enlarge the buffer size
from c++ side if necessary.


No, it's not.  ArrayBuffer sizes are immutable.


the algorithm first decodes base64 string into a binary string and
casts/copies it into the desired type based on the input
ArrayBufferView.


Does it cast or copy?  That is, does it do the moral equivalent of 
memcpy, or the moral equivalent of a for loop setting a[i] = b[i]?


-Boris