Re: [whatwg] canvas, img, file api and blobs

[Maciej Stachowiak]

On Feb 15, 2010, at 1:53 PM, Jonas Sicking wrote:

On Mon, Feb 15, 2010 at 6:43 AM, Stef Epardaud   
wrote:

Hello,

I am trying to write a client-side application in HTML5 that resizes
images before uploading them to the server. I saw several demos  
that did

this resizing using canvas and img, but I have only seen how to get a
data URL out of a canvas, and since Base64 is about 1.37% larger than
the equivalent binary data (according to http://en.wikipedia.org/wiki/Base64 
),

I wonder why it is not possible to get a Blob out of the canvas?
Especially since the File API supports uploading Blobs.


It has been suggested earlier on this list that we add a

canvas.toFile
or
canvas.toBlob

function, which would return a binary Blob instead of a data URL. For
exactly the reasons that you mention. I hope to write a test
implementation of this for firefox in the near future.


I still think we need a type for binary data that can represent in- 
memory resources. Blobs only allow asynchronous access, since they are  
meant to represent something like a file, a slice of a file, or a  
chunk of data in an on-disk database. It doesn't make sense to use  
such an inconvenient interface to pull out the contents of the canvas,  
which pretty much have to already be in memory.


Even something like WebGL's typed arrays would be better, if the  
ECMAScript committee doesn't come up with a good solution for basic  
binary data soon.


Regards,
Macie

Re: [whatwg] Video source selection based on quality (was: feedback)

[Tim Hutt]

On 16 February 2010 04:44, Hugh Guiney  wrote:
> While it is true that the amount of information in the SOURCE image
> does not change, the amount of information in the RESULT image *does*,
> simply by nature of the fact that it is no longer the same image.

Wrong wrong wrong wrong wrong. Come on, I think you know that
interpolation doesn't increase the amount of information available.
You're just being argumentative.

Youtube's UI is absolutely fine for its intended purpose. In *most*
cases, higher resolution = higher quality, and youtube know that 720p
and 1080p *are* meaningful to the average user (thanks to HDTV
advertising - which is also why they've kept the p suffix).

Anyway, with respect to the actual discussion. My vote is to add two
optional tags to : bitrate="800" (in kb/s) and
resolution="640x480" (native resolution). Then leave it up to the UA
if they want to show a UI to choose, or pick automatically.

Re: [whatwg] Video source selection based on quality (was: feedback)

[Hugh Guiney]

On Tue, Feb 16, 2010 at 12:35 AM, Gregory Maxwell  wrote:
>
> ...
>
> Pixel aspect ratios.
>
> This whole discussion has been painful to watch.

I know what pixel aspect ratios are. All too well, actually—for
instance, the square-pixel equivalent of 720x480 widescreen can be
either 853x480, 854x480, 856x480, 864x480, 873x480, or 875x480
depending on whom you ask. What fun! I actually attempted to explain
this in another thread:


So, yes, I realize that those dimensions are PAR-compensated. Most
users don't, which is why the shorthand form doesn't do them any
favors.

Re: [whatwg] Video source selection based on quality (was: feedback)

[Gregory Maxwell]

On Mon, Feb 15, 2010 at 11:44 PM, Hugh Guiney  wrote:
> And when other established terms are used, like "480p"—which, in
> virtually every other context, refers to 720x480, the most common of
> the acceptable resolution for DVDs—yet the video is *854*x480, that's
> also confusing. Ted could download such a "480p" clip and attempt to
> burn it, only to discover that his authoring program won't accept that
> format.

...

Pixel aspect ratios.

This whole discussion has been painful to watch.

Re: [whatwg] Video source selection based on quality (was: feedback)

[Hugh Guiney]

On Mon, Feb 15, 2010 at 8:09 PM, David Singer  wrote:
> I think I agree with Tim here.  When you ask to watch "360p" content, you are 
> asking for content that has 360 lines of pixels to be displayed to you.

Right.

> You're not asking for whatever is displayed to you to occupy only 360 lines 
> of pixels on your display.

That's what YouTube's doing, though. In compacted mode, YouTube
displays *everything* at 640x360. In expanded mode, everything is
displayed at 854x480. That's regardless of whether you select "360p",
"480p", "720p", or "1080p"—which are all *supposed* to be
abbreviations for different frame sizes.

What you are *actually* selecting is which of the multiple encodes
YouTube automatically creates of your video to act as the source
video, which is then scaled to 640x360 or 854x480 depending on the
selected player size.

Using the [pixel height][scan type] notation in this way is
misleading, since they're no longer semantic (UNLESS you know what's
going on behind-the-scenes, which most people don't). Video is already
very confusing and labels like that, with no explanation, don't help
matters.

> Yes, when it is shown larger, then filtering etc. is done to avoid 
> pixel-doubling blocky artifacts;  this does not increase (and, we hope, not 
> decrease) the amount of information in the scene.

Interpolation is not "pixel-doubling"—it can be done in that way
(i.e., where every source pixel is mapped to a multiple of itself),
but there are many different algorithms that determine how an image is
to be scaled and they all work differently with varying levels of
visual degradation and processor intensity.

While it is true that the amount of information in the SOURCE image
does not change, the amount of information in the RESULT image *does*,
simply by nature of the fact that it is no longer the same image.

This is far and away from what I was originally discussing though. If
you want to learn more about how image interpolation works this is a
good primer: 


> With the advent of higher-resolution displays, and the ability to use CSS 
> with HTML to set 'sensible' sizes of video relative to other page content, 
> the assumption that video will always be displayed in a 1:1 ratio of source 
> (coded, transmitted) pixel to display pixel is increasingly untenable.

I never said that video will always be displayed 1:1; that's
completely unrealistic. Every TV currently in production scales the
incoming image in some way with default settings. This is because most
consumers can not cope with letterboxing and manufacturers need a way
to sell bigger and bigger screen sizes even even when the content
doesn't fit snugly between each corner. What I was saying was that we
shouldn't refer to something by a name that means something else.

Yes, if you select 1080p on an HD YT clip, it'll use the 1080p
file—but it WON'T fill up Joe's brand new 1080p computer monitor until
he goes into fullscreen mode. But how is he supposed to know that he
has to do that? It SAYS 1080p down at the bottom, after all. For all
he knows that's the size of the video. Similarly, Jane may be thinking
about buying a new monitor herself, but after seeing how "small" 1080p
is on YT, decides it's not worth it.

And when other established terms are used, like "480p"—which, in
virtually every other context, refers to 720x480, the most common of
the acceptable resolution for DVDs—yet the video is *854*x480, that's
also confusing. Ted could download such a "480p" clip and attempt to
burn it, only to discover that his authoring program won't accept that
format.

What is boils down to is that video formats are *already* very
confusing, and that fact does not need to be compounded by careless
labeling.

-Hugh

Re: [whatwg] Video source selection based on quality (was: feedback)

[David Singer]

I think I agree with Tim here.  When you ask to watch "360p" content, you are 
asking for content that has 360 lines of pixels to be displayed to you.  You're 
not asking for whatever is displayed to you to occupy only 360 lines of pixels 
on your display. Yes, when it is shown larger, then filtering etc. is done to 
avoid pixel-doubling blocky artifacts;  this does not increase (and, we hope, 
not decrease) the amount of information in the scene.

With the advent of higher-resolution displays, and the ability to use CSS with 
HTML to set 'sensible' sizes of video relative to other page content, the 
assumption that video will always be displayed in a 1:1 ratio of source (coded, 
transmitted) pixel to display pixel is increasingly untenable.

On Feb 15, 2010, at 17:03 , Hugh Guiney wrote:

> On Mon, Feb 15, 2010 at 7:07 PM, Tim Hutt  wrote:
>> Erm, what? The 360p refers to the 'native' resolution of the video
>> file youtube sends. If you play a 360p video fullscreen, it's still
>> only got 360 lines; they're just scaled up. It would be meaningless if
>> the number referred to the final playback size since that is
>> independent of the video quality.
> 
> By "lines", do you mean TV (or scan) lines?
> 
> TV lines and pixels can both be used to describe image resolution, but
> they aren't the same thing. The difference is very technical and isn't
> relevant to this conversation but essentially, TV lines pertain to
> analog systems and pixels pertain to digital systems. And in the
> digital realm, magnification is achieved through interpolation: since
> the source image has less pixels than the destination image, a
> resizing algorithm must invent pixels to fill in all of the unknown
> values in the target image, based on the known values in the source
> image. The result is a "best guess" of what the image might look like
> at that resolution. The pixel count has changed; it is therefore new
> data and not the same as the input.
> 
> The way YouTube has it now is meaningless since the player doesn't
> expand when you change the height (that's what that number is supposed
> to indicate). The older standard/HQ/HD made more sense.

David Singer
Multimedia and Software Standards, Apple Inc.

Re: [whatwg] Video source selection based on quality (was: feedback)

[Hugh Guiney]

On Mon, Feb 15, 2010 at 7:07 PM, Tim Hutt  wrote:
> Erm, what? The 360p refers to the 'native' resolution of the video
> file youtube sends. If you play a 360p video fullscreen, it's still
> only got 360 lines; they're just scaled up. It would be meaningless if
> the number referred to the final playback size since that is
> independent of the video quality.

By "lines", do you mean TV (or scan) lines?

TV lines and pixels can both be used to describe image resolution, but
they aren't the same thing. The difference is very technical and isn't
relevant to this conversation but essentially, TV lines pertain to
analog systems and pixels pertain to digital systems. And in the
digital realm, magnification is achieved through interpolation: since
the source image has less pixels than the destination image, a
resizing algorithm must invent pixels to fill in all of the unknown
values in the target image, based on the known values in the source
image. The result is a "best guess" of what the image might look like
at that resolution. The pixel count has changed; it is therefore new
data and not the same as the input.

The way YouTube has it now is meaningless since the player doesn't
expand when you change the height (that's what that number is supposed
to indicate). The older standard/HQ/HD made more sense.

Re: [whatwg] Video source selection based on quality (was: feedback)

[Tim Hutt]

On 15 February 2010 23:07, Hugh Guiney  wrote:
> But even if we had a standard, YouTube further dilutes the meaning of
> these abbreviations since they now also have a toggle button (depicted
> as two arrows at a right angle) that expands or contracts the player
> but leaves the quality setting the same. So if you select "360p", and
> decide you want it to fill more of your screen, it will, but then it's
> no longer 360 pixels tall because it's been scaled.

Erm, what? The 360p refers to the 'native' resolution of the video
file youtube sends. If you play a 360p video fullscreen, it's still
only got 360 lines; they're just scaled up. It would be meaningless if
the number referred to the final playback size since that is
independent of the video quality.

Re: [whatwg] Video source selection based on quality (was: feedback)

[Hugh Guiney]

Thanks for your insight Silvia.

On Wed, Feb 10, 2010 at 2:47 AM, Silvia Pfeiffer
 wrote:
> Firstly, I think that explicit user choice isn't a problem.
>
> As a content provider, you have several means of doing this user choice:
>
> 1) You can provide in a single (admittedly javascript-based) video
> player interface an option to the user to switch between source files
> of different quality (bitrate, width x height, audio samplerate, and
> whatever other choices you make for differently encoded content).
> This is what YouTube does in their latest players, e.g. 360p / 480p / 720p
> choice (though this is not really a quality measure, but only a
> measure of width x height, but since the display size is not changed
> in YouTube, it actually is a quality setting).

I can *maybe* see this feature being a video player UI component (more
on why in a bit), though not a JS-based one. I imagine people with
slower computers/connections and/or in more restrictive environments,
who probably stand to benefit the most from this, would be more likely
to have JS off. Additionally, it would require document authors to
take on the responsibility of scripting their own content selection
algorithms (unless there's a standard library that everyone just
copies and pastes), which seems unnecessary given the fact that
resource selection is already capable of being done by the browser
and/or server.

Also, the choices don't *only* measure width x height, but also
indicate of how frames are scanned: "p" for progressive and "i" for
interlaced, which have a direct impact on both file size and perceived
image quality. Although, the label "p" is redundant in YouTube, since
AFAIK, it automatically de-interlaces whatever you upload.

The problems with making this a video UI component are that:

It'd be heavily abbreviated. YouTube is using an industry convention
that has only entered consumer parlance due to HDTV marketing (just
Google "720p vs. 1080i" or "1080p TrueHD"), which ONLY specifies
height and scan type, since the rest of the information is implied due
to engineering and broadcast standards, e.g. 1080p implies 1920x1080
29.97 or 23.976 fps progressive scan; 720p implies the same but at
1280x720. But "360p" doesn't imply anything, because there's no
standard that defines it, and once you hit the SD level (480p and
below), there are two different display sizes depending on the pixel
aspect ratio.

But even if we had a standard, YouTube further dilutes the meaning of
these abbreviations since they now also have a toggle button (depicted
as two arrows at a right angle) that expands or contracts the player
but leaves the quality setting the same. So if you select "360p", and
decide you want it to fill more of your screen, it will, but then it's
no longer 360 pixels tall because it's been scaled.

The alternative would be to specify the video information in full, or
in a partially-abbreviated form. But then you'd have to cram stuff
like "1920x1080p24 (Scaled to 1280x720)" into the UI, which crowds the
other controls and hinders the viewing experience.

The other thing is that so much goes into video. Yes,

> (bitrate, width x height, audio samplerate

go into it, but

> whatever other choices you make for differently encoded content).

covers a huge spectrum, as I previously outlined in the original
thread: 
.

All of this criteria is potentially useful, so maybe the goal for now
should be to prioritize them in the order of importance they'd be to
the average user and implement a high-ranking subset.

> 2) If users log in to use your content, you can ask them to provide a
> default setting, just like YouTube does it in their Account Settings
> (as Hugh describes below).

This may be fine for video portal sites, but not every page utilizes
logins. Most people just want to share a video they made or like with
their audience, the same way they would an image. And they may be
using a free blogging service that doesn't allow them to implement
additional features. I also find it impractical to require a login
system to be in place just to ask users to select a content quality
preference.

> 3) Even if users don't log in, you can have a button on the side of
> the video (and a once-off splashscreen if you so like), which allows
> users to set and change their preference and leave a cookie to
> remember their choice.

I'd be OK with cookies as an interim solution but they're not ideal,
since they'd require setting new preferences for every site visited
while the clients' connection and computational speeds would stay more
or less the same.

> All of this is based on the premise that the user either knows what
> their pipe and their computer can take, or experiments with it until
> he/she is happy.

Well, they don't always have to know off the top of their head. P2P
programs often allow one to run automated tests to estimate the speed
of his/her current connectio

Re: [whatwg] canvas, img, file api and blobs

[Jonas Sicking]

On Mon, Feb 15, 2010 at 6:43 AM, Stef Epardaud  wrote:
> Hello,
>
> I am trying to write a client-side application in HTML5 that resizes
> images before uploading them to the server. I saw several demos that did
> this resizing using canvas and img, but I have only seen how to get a
> data URL out of a canvas, and since Base64 is about 1.37% larger than
> the equivalent binary data (according to http://en.wikipedia.org/wiki/Base64),
> I wonder why it is not possible to get a Blob out of the canvas?
> Especially since the File API supports uploading Blobs.

It has been suggested earlier on this list that we add a

canvas.toFile
or
canvas.toBlob

function, which would return a binary Blob instead of a data URL. For
exactly the reasons that you mention. I hope to write a test
implementation of this for firefox in the near future.

> In a related question, is there any guarantee that when we draw an image
> with EXIF information into a canvas for resizing, we get (or not) the
> EXIF back in the resized image (currently via toDataURL())?

Like Boris says, no, copying an image into a canvas doesn't copy the
EXIF data. It would make sense to add some API to canvas that allows
importing metadata from a given image.

/ Jonas

Re: [whatwg] canvas, img, file api and blobs

[Stef Epardaud]

On Mon, Feb 15, 2010 at 05:16:43PM +0100, Anne van Kesteren wrote:
> >You don't.  The canvas is just a bitmap; it has no attached metadata.
> >
> >This could be changed, but various things would need to be defined about  
> >what happens when multiple images are drawn in, when images are drawn on  
> >top of other things in the canvas, when stuff is drawn on top of images  
> >in the canvas, etc.

Very true. Doesn't look right to expect EXIF out of a canvas.

> Seems easier to take care of this by allowing extraction of metadata (some  
> W3C WG is working on this I think) and maybe a way to put it back in...

That would be very welcome, the Blob type doesn't even seem to have a
way to iterate or modify the contents, which would be required if we
were to manually scan the original metadata and move it to the resized
image.
-- 
Stéphane Epardaud

Re: [whatwg] canvas, img, file api and blobs

[Anne van Kesteren]

On Mon, 15 Feb 2010 17:10:30 +0100, Boris Zbarsky  wrote:

On 2/15/10 9:43 AM, Stef Epardaud wrote:

In a related question, is there any guarantee that when we draw an image
with EXIF information into a canvas for resizing, we get (or not) the
EXIF back in the resized image (currently via toDataURL())?


You don't.  The canvas is just a bitmap; it has no attached metadata.

This could be changed, but various things would need to be defined about  
what happens when multiple images are drawn in, when images are drawn on  
top of other things in the canvas, when stuff is drawn on top of images  
in the canvas, etc.


Seems easier to take care of this by allowing extraction of metadata (some  
W3C WG is working on this I think) and maybe a way to put it back in...



--
Anne van Kesteren
http://annevankesteren.nl/

Re: [whatwg] canvas, img, file api and blobs

[Boris Zbarsky]

On 2/15/10 9:43 AM, Stef Epardaud wrote:

In a related question, is there any guarantee that when we draw an image
with EXIF information into a canvas for resizing, we get (or not) the
EXIF back in the resized image (currently via toDataURL())?


You don't.  The canvas is just a bitmap; it has no attached metadata.

This could be changed, but various things would need to be defined about 
what happens when multiple images are drawn in, when images are drawn on 
top of other things in the canvas, when stuff is drawn on top of images 
in the canvas, etc.


-Boris

[whatwg] canvas, img, file api and blobs

[Stef Epardaud]

Hello,

I am trying to write a client-side application in HTML5 that resizes
images before uploading them to the server. I saw several demos that did
this resizing using canvas and img, but I have only seen how to get a
data URL out of a canvas, and since Base64 is about 1.37% larger than
the equivalent binary data (according to http://en.wikipedia.org/wiki/Base64),
I wonder why it is not possible to get a Blob out of the canvas?
Especially since the File API supports uploading Blobs.

In many cases it is useful to resize images on the client so that upload
size can be limited (to save bandwidth) and uploading in base64 would
mitigate this.

In a related question, is there any guarantee that when we draw an image
with EXIF information into a canvas for resizing, we get (or not) the
EXIF back in the resized image (currently via toDataURL())?

Thanks
-- 
Stéphane Epardaud

Re: [whatwg] prompts, alerts and showModalDialog during beforeunload/unload events

[Tab Atkins Jr.]

On Mon, Feb 15, 2010 at 2:29 AM, Biju  wrote:
> I dont know whether you all saw list of people on
> https://bugzilla.mozilla.org/show_bug.cgi?id=391834
> https://bugzilla.mozilla.org/show_bug.cgi?id=61098

Those two address a *completely* separate issue, that of someone
running an infinite alert() loop.  They can do that anywhere they
want, and it has the same effect as in the beforeunload handler.
Firefox just doesn't handle that well - Opera and Chrome, iirc,
provide an option to ignore further modal dialogs from the domain once
they've popped up several in a short amount of time.

~TJ

Re: [whatwg] <% text %> and in corporate intranet html content

[Tab Atkins Jr.]

On Mon, Feb 15, 2010 at 2:45 AM, Biju  wrote:
> Also as a user I feel why cant Firefox, Safari, Opera, Chrome agree to
> behave same.

Because, before HTML5, they all had to guess at what the others were
doing.  ^_^  As they all implement the HTML5 parsing algorithm, they
will act the same.

~TJ

Re: [whatwg] prompts, alerts and showModalDialog during beforeunload/unload events

[Mike Hearn]

Browsers could solve the editor use case by treating "close tab" as
"hide tab" for a minute or two before actually shutting down the page.

Then the problem becomes, how do you make it obvious to users that
they can get their work back by pressing a magic button somewhere?

The modal quit loop is frequently used to try and make people download
malware. It'd be nice to eliminate it in a backwards compatible
manner.

Re: [whatwg] <% text %> and in corporate intranet html content

[Biju]

Why cant we consider <% some text %> and  as
pre-processor command node in HTML DOM.

Also as a user I feel why cant Firefox, Safari, Opera, Chrome agree to
behave same.

Re: [whatwg] prompts, alerts and showModalDialog during beforeunload/unload events

[Biju]

I dont know whether you all saw list of people on
https://bugzilla.mozilla.org/show_bug.cgi?id=391834
https://bugzilla.mozilla.org/show_bug.cgi?id=61098

Re: [whatwg] <% text %> and in corporate intranet html content

[Gordon P. Hemsley]

On Tue, Feb 9, 2010 at 10:05 PM, Biju  wrote:

> What should a user agent display when html content is...
>
> 
> <%@ page language="java" %>
> 
>
> At present IE and Safari display blank
>
> Firefox display <%@ page language="java" %>
>
> And for document.body.innerHTML browsers give
> Firefox --> <%@ page language="java" %>
> IE --><%@ page language="java" %>
> and Safari gives blank
>
> Also for
> 
> 
> 
>
> Firefox gives blank
>
> But for
> 
> abc "  ?> xyz
> 
>
> Firefox display...
> abc " ?> xyz
>
> ie, all the contents after first ">"
> with .innerHTML --> abc "  ?> xyz
>
> IE in this case again hide all content till "?>"
> as well as preserve content including the white space in innerHTML
>
> Due to these problems browsing corporate intranet with Firefox is
> little irritating.
> Calling help desk and asking to provide fix will get a reply that
> company has standardized on IE6, so please use IE.
>
>
> So per HTML standard in both case what should user agent display and
> as well as content of .innerHTML
>
> Thanks
> Biju
>

For what it's worth, I filed a Mozilla bug on a similar issue, and it was
marked INVALID.

https://bugzilla.mozilla.org/show_bug.cgi?id=477455
Parser does not wait for "?>" to close blocks that begin with "http://gphemsley.org/ • http://gphemsley.org/blog/
http://sasha.sourceforge.net/ • http://www.yoursasha.com/