Re: [whatwg] Encoding: API

[Anne van Kesteren]

On Thu, Oct 11, 2012 at 6:09 AM, Anne van Kesteren  wrote:
> On Wed, Oct 10, 2012 at 7:28 PM, Joshua Bell  wrote:
>> Practically speaking, this would mean refactoring the combined spec so that
>> the current BOM handling is defined for parsing web content outside of the
>> API rather than requiring the API to hack around it.
>
> You would still get the hack because the API requires special
> treatment for "utf-16". Given that per Unicode "utf-16le" and
> "utf-16be" outlaw the BOM, maybe a good solution would be a flag to
> disable BOM handling as seen by the decode algorithm? So the decoder
> gets a disableBOM flag that defaults to false? That would only require
> a special case for BOM handling on top of what there is today, which
> seems a fair bit cleaner.

The main problem with this is that you would get a leading BOM in
utf-8 if the content includes that. An unlikely scenario, but maybe we
want to take care of that. Another approach I thought about is that we
have an "API decode" algorithm, which is very similar to

http://encoding.spec.whatwg.org/#decode

However, instead of setting the encoding, it checks if the leading
bytes match, and if the encoding matches, and only then does it set
the offset. So the BOM would be skipped for utf-8/utf-16 if it was a
valid BOM, but a BOM invalid for the given encoding would never switch
the encoding.

The behavior of the normal decode algorithm does not need to be
exposed through the API I think, unless a use case comes up at some
point.


-- 
http://annevankesteren.nl/

Re: [whatwg] Features for responsive Web design

[Mark Callow]

On 2012/10/10 12:29, Ian Hickson wrote:
> On Wed, 10 Oct 2012, Mark Callow wrote:
>> I don't know what the browser on the SH-10D is doing, (It's running 
>> Android 4.0) but, given the physical size (4.5"), if they were making 
>> the css pixels smaller, the content would be unreadable. I expect they 
>> are actually using 3x.
> Can you obtain a screenshot of this page in the device's browser?
>
>http://junkyard.damowmow.com/513
>
> (Should be power+voldown to get a screen shot.)

I don't have one  of these phones. I went to the Docomo shop on the way
home yesterday and was able to view the page on it but could not get or
send a screenshot.

The size of the black rectangle relative to the cats image looks
slightly smaller than on my PC and I'd say the right edge of it is
further to the left of the image than on my PC but overall the result is
broadly similar.

Regular  text on web pages is very small. It really isn't practical to
read it without zooming in. So I suspect they are using 2x thus making
CSS pixels smaller. There's probably a certain amount of chicken & egg
here. How much content would break on devices with a 3:1 device:css
ratio? That concern probably stops device makers setting 3x even when
they really should.

Regards

-Mark


-- 
注意：この電子メールには、株式会社エイチアイの機密情報が含まれている場合
が有ります。正式なメール受信者では無い場合はメール複製、 再配信または情
報の使用を固く禁じております。エラー、手違いでこのメールを受け取られまし
たら削除を行い配信者にご連絡をお願いいたし ます.

NOTE: This electronic mail message may contain confidential and
privileged information from HI Corporation. If you are not the intended
recipient, any disclosure, photocopying, distribution or use of the
contents of the received information is prohibited. If you have received
this e-mail in error, please notify the sender immediately and
permanently delete this message and all related copies.

Re: [whatwg] Features for responsive Web design

[Maciej Stachowiak]

On Oct 10, 2012, at 1:14 AM, Maciej Stachowiak  wrote:

> 
> On Oct 9, 2012, at 2:49 PM, Tab Atkins Jr.  wrote:
> 
>> On Tue, Oct 9, 2012 at 11:48 AM, Ian Hickson  wrote:
>>> On Tue, 9 Oct 2012, Mark Callow wrote:
 On 2012/10/06 7:09, Ian Hickson wrote:
> I agree, when there's 3x displays, this could get to the point where we
> need to solve it. :-)
 
 With the current displays, it's just not that big a deal, IMHO. If by 3x
 you mean displays whose dpi is 3x that of CSS pixels (96dpi), they
 already exist in retail products. I saw 2 last week.
>>> 
>>> Can you elaborate?
>>> 
>>> How many device pixels per CSS pixel do browsers on those devices use? Are
>>> they just making CSS pixels smaller, or are they actually using 3x?
>> 
>> http://www.zdnet.com/google-nexus-10-tablet-to-have-higher-res-display-than-ipad-705466/
>> appears to be 299dpi
>> http://www.iclarified.com/entry/index.php?enid=3 appears to be 440dpi
>> 
>> These devices aren't out yet, but I suspect browsers would be
>> more-or-less as high-dpi as possible.
> 
> This page lists several devices with physical DPI higher than 288 (3x the 
> nominal CSS dpi) but none with a CSS pixel ratio greater than 2x. (To be 
> fair, the data is incomplete and may be inaccurate, though to my knowledge 
> the entries for Apple devices are all correct). So it's not a given that the 
> cited hardware dpi values would lead to higher CSS pixel ratios in the 
> corresponding software.

Apparently I forgot to link the page I meant: 


Regards,
Maciej

Re: [whatwg] Encoding: API

[Anne van Kesteren]

On Wed, Oct 10, 2012 at 7:28 PM, Joshua Bell  wrote:
> On Wed, Oct 10, 2012 at 6:42 AM, Anne van Kesteren  wrote:
>> I also still think it's kinda yucky that this API has this gigantic
>> hack around what the rest of the platform does with respect to the
>> byte order mark. It seems really weird to not expose the same
>> encode/decode that HTML/XML/CSS/etc. use.
>
> IMHO the API needs to support use cases: (1) code that wants to follow the
> behavior of the web platform with respect to legacy content (i.e. the
> desire to self-host), and (2) code that wants to parse files that are not
> traditionally "web" data, i.e. fragments of binary files, which don't have
> legacy behavior and where BOM taking priority would be surprising to
> developers. For #2, following the behavior of APIs like ICU with respect to
> BOMs is more sensible. I believe #2 is higher priority as long as it does
> not preclude #1, and #1 can be achieved by code that inspects the stream
> before handing it off to the decoder.
>
> Practically speaking, this would mean refactoring the combined spec so that
> the current BOM handling is defined for parsing web content outside of the
> API rather than requiring the API to hack around it.

You would still get the hack because the API requires special
treatment for "utf-16". Given that per Unicode "utf-16le" and
"utf-16be" outlaw the BOM, maybe a good solution would be a flag to
disable BOM handling as seen by the decode algorithm? So the decoder
gets a disableBOM flag that defaults to false? That would only require
a special case for BOM handling on top of what there is today, which
seems a fair bit cleaner.


> I received feedback recently that the API is perhaps too terse right now
> when dealing with streaming content, and a more explicit decode(),
> decodeStream(), resetStream() might be more intelligible. Thoughts?

Either way works for me.


-- 
http://annevankesteren.nl/

Re: [whatwg] Encoding: API

[Jonas Sicking]

On Wed, Oct 10, 2012 at 6:42 AM, Anne van Kesteren  wrote:
> Hey, I was wondering whether it would make sense to define
> http://wiki.whatwg.org/wiki/StringEncoding as part of
> http://encoding.spec.whatwg.org/ Tying them together makes sense to me
> anyway and is similar to what we do with URL, HTML, etc.
>
> As for the open issue, I think it would make sense if the encoding's
> name was returned. Label is just some case-insensitive keyword to get
> there.
>
> I also still think it's kinda yucky that this API has this gigantic
> hack around what the rest of the platform does with respect to the
> byte order mark. It seems really weird to not expose the same
> encode/decode that HTML/XML/CSS/etc. use.

Also, Firefox 18 as well as recent nightlies implement this draft. It
should be matching the draft pretty closely, though I know there is
some weirdness in handling invalid input for some encodings when doing
decoding. This is also hooked up to the generic Gecko decoder backend
which means that the decoder doesn't yet support the exact set of
encodings as defined by the http://encoding.spec.whatwg.org/ spec.

/ Jonas

Re: [whatwg] Control and Undefined Characters

[Ian Hickson]

On Thu, 11 Oct 2012, Cameron Zemek wrote:
> 
> So section '3.2.5.1.5 Phrasing content' is only referring to conforming 
> documents when it mentions that text nodes and attribute values should 
> not include control characters other than space characters?

It's refering to the conformance rules for documents, yes.

This section in the intro talks about this issue:

   
http://www.whatwg.org/specs/web-apps/current-work/multipage/introduction.html#how-to-read-this-specification

HTH,
-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Control and Undefined Characters

[Cameron Zemek]

On Thu, Oct 11, 2012 at 9:07 AM, Ian Hickson  wrote:
> User agents are required to treat U+0001 the same as, say, "A".

Yeah that is how I understood the specification.

>> And testing in Firefox and Chrome it appears these characters are
>> ignored. But I see no mention of this anywhere to ignore them or how to
>> handle them.
>
> Do you have a test case demonstrating this? When I tested it it seemed
> like the characters were not ignored:
>
>http://software.hixie.ch/utilities/js/live-dom-viewer/?saved=1824

Oh nevermind me. Saving the page and using a hex editor I see that it
still is there. Sorry I was inspecting the DOM and the control
characters are invisible characters (early in the morning and my brain
is not functioning correctly).

So section '3.2.5.1.5 Phrasing content' is only referring to
conforming documents when it mentions that text nodes and attribute
values should not include control characters other than space
characters?

Re: [whatwg] Control and Undefined Characters

[Ian Hickson]

On Thu, 11 Oct 2012, Cameron Zemek wrote:
>
> The spec states:
> "Any occurrences of any characters in the ranges U+0001 to U+0008,
> U+000E to U+001F, U+007F to U+009F, U+FDD0 to U+FDEF, and characters
> U+000B, U+FFFE, U+, U+1FFFE, U+1, U+2FFFE, U+2, U+3FFFE,
> U+3, U+4FFFE, U+4, U+5FFFE, U+5, U+6FFFE, U+6,
> U+7FFFE, U+7, U+8FFFE, U+8, U+9FFFE, U+9, U+AFFFE,
> U+A, U+BFFFE, U+B, U+CFFFE, U+C, U+DFFFE, U+D,
> U+EFFFE, U+E, U+E, U+F, U+10FFFE, and U+10 are parse
> errors. These are all control characters or permanently undefined
> Unicode characters (noncharacters)."
> 
> Additionally character references for these codepoints also will
> return these unicode characters. Therefore these characters are passed
> to the tree construction stage as far as I can tell. And I so no
> handling of them in the tree contruction.
> 
> Elsewhere in the specification it says:
> "Text nodes and attribute values must consist of Unicode characters,
> must not contain U+ characters, must not contain permanently
> undefined Unicode characters (noncharacters), and must not contain
> control characters other than space characters."

All these requirements relate to authoring conformance criteria and 
validators.

User agents are required to treat U+0001 the same as, say, "A".


> And testing in Firefox and Chrome it appears these characters are 
> ignored. But I see no mention of this anywhere to ignore them or how to 
> handle them.

Do you have a test case demonstrating this? When I tested it it seemed 
like the characters were not ignored:

   http://software.hixie.ch/utilities/js/live-dom-viewer/?saved=1824

(This test is testing whether a U+0001 is lost either in the JS parser, 
document.write(), the HTML tokeniser, the HTML parser, the DOM API, or the 
JS string API, and it seems to get through all of those fine.)

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

[whatwg] Control and Undefined Characters

[Cameron Zemek]

The spec states:
"Any occurrences of any characters in the ranges U+0001 to U+0008,
U+000E to U+001F, U+007F to U+009F, U+FDD0 to U+FDEF, and characters
U+000B, U+FFFE, U+, U+1FFFE, U+1, U+2FFFE, U+2, U+3FFFE,
U+3, U+4FFFE, U+4, U+5FFFE, U+5, U+6FFFE, U+6,
U+7FFFE, U+7, U+8FFFE, U+8, U+9FFFE, U+9, U+AFFFE,
U+A, U+BFFFE, U+B, U+CFFFE, U+C, U+DFFFE, U+D,
U+EFFFE, U+E, U+E, U+F, U+10FFFE, and U+10 are parse
errors. These are all control characters or permanently undefined
Unicode characters (noncharacters)."

Additionally character references for these codepoints also will
return these unicode characters. Therefore these characters are passed
to the tree construction stage as far as I can tell. And I so no
handling of them in the tree contruction.

Elsewhere in the specification it says:
"Text nodes and attribute values must consist of Unicode characters,
must not contain U+ characters, must not contain permanently
undefined Unicode characters (noncharacters), and must not contain
control characters other than space characters."

And testing in Firefox and Chrome it appears these characters are
ignored. But I see no mention of this anywhere to ignore them or how
to handle them.

Is this a bug with the specification?

Re: [whatwg] Features for responsive Web design

[Ian Hickson]

On Wed, 10 Oct 2012, Tim Kadlec wrote:
> >
> > That's actually exactly why it's better _not_ to plan for it. We can't 
> > design features for problems we don't understand. It's better to wait 
> > until we have real problems before fixing them.
> 
> You may not be able to predict every future problem, but surely you need 
> to keep it in mind as you create solutions for today, right?

Sure, that's why for example the srcset="" syntax is extensible and 
already supports arbitrary densities, not just 1x and 2x.


> For example, if all it takes is one higher resolution or one more 
> feature to come along before a solution becomes unwieldy doesn't that 
> imply the solution isn't a particularly strong one and is instead merely 
> a stopgap?

Only if we think that 3x is more likely to occur than not. It would be 
easy to support everything we could think of, even the unlikely things, 
but the cost would be that instead of a strong solution we'd have an 
unusable one.

In the world of perpetual progress, every solution is "stopgap". That 
doesn't mean they're not strong solutions in the meantime.


> We can't be too bold with our predictions, but we do have to build with 
> the future in mind or else condemn ourselves to a perpetual game of 
> catch-up.

Standardisation is the process of taking innovative ideas and describing 
them in an interoperably implementable way. Standardisation is by 
definition a perpetual game of catch-up.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Features for responsive Web design

[Tim Kadlec]

> That's actually exactly why it's better _not_ to plan for it. We can't
> design features for problems we don't understand. It's better to wait
> until we have real problems before fixing them.

You may not be able to predict every future problem, but surely you need to
keep it in mind as you create solutions for today, right? For example, if
all it takes is one higher resolution or one more feature to come along
before a solution becomes unwieldy doesn't that imply the solution isn't a
particularly strong one and is instead merely a stopgap?

We can't be too bold with our predictions, but we do have to build with the
future in mind or else condemn ourselves to a perpetual game of catch-up.

Take care,
Tim

-
http://twitter.com/tkadlec
http://timkadlec.com
http://implementingresponsivedesign.com

Re: [whatwg] Features for responsive Web design

[Ian Hickson]

On Wed, 10 Oct 2012, Mathew Marquis wrote:
> 
> In fairness, no, we can’t predict the future one way or the other. 
> That’s exactly why it’s better to plan for it than not.

That's actually exactly why it's better _not_ to plan for it. We can't 
design features for problems we don't understand. It's better to wait 
until we have real problems before fixing them.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Encoding: API

[Joshua Bell]

On Wed, Oct 10, 2012 at 6:42 AM, Anne van Kesteren  wrote:

> Hey, I was wondering whether it would make sense to define
> http://wiki.whatwg.org/wiki/StringEncoding as part of
> http://encoding.spec.whatwg.org/ Tying them together makes sense to me
> anyway and is similar to what we do with URL, HTML, etc.
>

No objection from me.


> As for the open issue, I think it would make sense if the encoding's
> name was returned. Label is just some case-insensitive keyword to get
> there.
>

I tend to agree, as the label gives you no information you don't already
have and the name can be at least a diagnostic.


> I also still think it's kinda yucky that this API has this gigantic
> hack around what the rest of the platform does with respect to the
> byte order mark. It seems really weird to not expose the same
> encode/decode that HTML/XML/CSS/etc. use.
>

IMHO the API needs to support use cases: (1) code that wants to follow the
behavior of the web platform with respect to legacy content (i.e. the
desire to self-host), and (2) code that wants to parse files that are not
traditionally "web" data, i.e. fragments of binary files, which don't have
legacy behavior and where BOM taking priority would be surprising to
developers. For #2, following the behavior of APIs like ICU with respect to
BOMs is more sensible. I believe #2 is higher priority as long as it does
not preclude #1, and #1 can be achieved by code that inspects the stream
before handing it off to the decoder.

Practically speaking, this would mean refactoring the combined spec so that
the current BOM handling is defined for parsing web content outside of the
API rather than requiring the API to hack around it.

...

While we're here, any feedback from implementers? Mozilla is apparently
quite far along. Any surprises or additional issues? Any initial feedback
from users?

I received feedback recently that the API is perhaps too terse right now
when dealing with streaming content, and a more explicit decode(),
decodeStream(), resetStream() might be more intelligible. Thoughts?

[whatwg] Encoding: API

[Anne van Kesteren]

Hey, I was wondering whether it would make sense to define
http://wiki.whatwg.org/wiki/StringEncoding as part of
http://encoding.spec.whatwg.org/ Tying them together makes sense to me
anyway and is similar to what we do with URL, HTML, etc.

As for the open issue, I think it would make sense if the encoding's
name was returned. Label is just some case-insensitive keyword to get
there.

I also still think it's kinda yucky that this API has this gigantic
hack around what the rest of the platform does with respect to the
byte order mark. It seems really weird to not expose the same
encode/decode that HTML/XML/CSS/etc. use.


-- 
http://annevankesteren.nl/

Re: [whatwg] Features for responsive Web design

[Mathew Marquis]

On Oct 10, 2012, at 4:14 AM, Maciej Stachowiak  wrote:

> 
> On Oct 9, 2012, at 2:49 PM, Tab Atkins Jr.  wrote:
> 
>> On Tue, Oct 9, 2012 at 11:48 AM, Ian Hickson  wrote:
>>> On Tue, 9 Oct 2012, Mark Callow wrote:
 On 2012/10/06 7:09, Ian Hickson wrote:
> I agree, when there's 3x displays, this could get to the point where we
> need to solve it. :-)
 
 With the current displays, it's just not that big a deal, IMHO. If by 3x
 you mean displays whose dpi is 3x that of CSS pixels (96dpi), they
 already exist in retail products. I saw 2 last week.
>>> 
>>> Can you elaborate?
>>> 
>>> How many device pixels per CSS pixel do browsers on those devices use? Are
>>> they just making CSS pixels smaller, or are they actually using 3x?
>> 
>> http://www.zdnet.com/google-nexus-10-tablet-to-have-higher-res-display-than-ipad-705466/
>> appears to be 299dpi
>> http://www.iclarified.com/entry/index.php?enid=3 appears to be 440dpi
>> 
>> These devices aren't out yet, but I suspect browsers would be
>> more-or-less as high-dpi as possible.
> 
> This page lists several devices with physical DPI higher than 288 (3x the 
> nominal CSS dpi) but none with a CSS pixel ratio greater than 2x. (To be 
> fair, the data is incomplete and may be inaccurate, though to my knowledge 
> the entries for Apple devices are all correct). So it's not a given that the 
> cited hardware dpi values would lead to higher CSS pixel ratios in the 
> corresponding software.

No, but we do know that things are continuing to trend towards higher PPI 
displays, and that at some point that may lead to a higher CSS pixel ratio. As 
a member of the jQuery Mobile project I’ve seen this for myself with the test 
devices we’re receiving constantly—every new screen is sharper than the last.

In fairness, no, we can’t predict the future one way or the other. That’s 
exactly why it’s better to plan for it than not.

Re: [whatwg] Features for responsive Web design

[Maciej Stachowiak]

On Oct 9, 2012, at 2:49 PM, Tab Atkins Jr.  wrote:

> On Tue, Oct 9, 2012 at 11:48 AM, Ian Hickson  wrote:
>> On Tue, 9 Oct 2012, Mark Callow wrote:
>>> On 2012/10/06 7:09, Ian Hickson wrote:
 I agree, when there's 3x displays, this could get to the point where we
 need to solve it. :-)
>>> 
>>> With the current displays, it's just not that big a deal, IMHO. If by 3x
>>> you mean displays whose dpi is 3x that of CSS pixels (96dpi), they
>>> already exist in retail products. I saw 2 last week.
>> 
>> Can you elaborate?
>> 
>> How many device pixels per CSS pixel do browsers on those devices use? Are
>> they just making CSS pixels smaller, or are they actually using 3x?
> 
> http://www.zdnet.com/google-nexus-10-tablet-to-have-higher-res-display-than-ipad-705466/
> appears to be 299dpi
> http://www.iclarified.com/entry/index.php?enid=3 appears to be 440dpi
> 
> These devices aren't out yet, but I suspect browsers would be
> more-or-less as high-dpi as possible.

This page lists several devices with physical DPI higher than 288 (3x the 
nominal CSS dpi) but none with a CSS pixel ratio greater than 2x. (To be fair, 
the data is incomplete and may be inaccurate, though to my knowledge the 
entries for Apple devices are all correct). So it's not a given that the cited 
hardware dpi values would lead to higher CSS pixel ratios in the corresponding 
software.

Regards,
Maciej