Re: [websec] Issue 17: Registry for magic numbers

[Adam Barth]

You've posed a large number of questions.  I'll do my best to answer them.

On Tue, Oct 25, 2011 at 11:31 PM, Larry Masinter  wrote:
> This gets back to the question of the scope of the document. Does it, or does 
> it not, handle sniffing of arbitrary blobs of data that come in without any 
> content-type,

User agents that implement sniffing are expected to sniff HTTP
responses that lack a Content-Type header, yes.

> blobs of data labeled application/octet-stream,

User agents that implement sniffing are not expected to sniff HTTP
responses that contain Content-Type header with the value
application/octet-stream.

> and data coming via ftp (through ftp URIs)

Yes.

> or thumb drives

Yes.

>or mounted NFS file systems or whatever?

Yes.

You can answer these questions by reading the document.  For example,
the document explicitly states the set of Content-Type header values
that trigger sniffing.  The document also explicitly calls out FTP as
an example.

> Does it, or does it not, handle sniffing rules inside ZIP packaged web 
> applications?

Not as described by this document.  However, I've been told that
another document has re-used the algorithm for that purpose.

> If it does, then sniffing should cover everything that is sniffable, 
> including almost all MIME types

Why is that?  The document describes what is essentially the minimal
amount of sniffing a user agent needs to perform in order to be
competitive in the browser market.  I don't think we should be
encouraging sniffing beyond that.

> -- you say "most MIME types that get registered don't need sniffing rules", I 
> don't know what the percentage is,

With the possible exception of fonts, I believe the document describes
all the sniffing rules that are necessary today.  You can compare with
list of MIME types in the document with the list of registered MIME
types if you wish to get a sense of what I mean when I say that "most"
don't need sniffing rules.

> but after all, don't you want to be able to discover file types??

I'm not sure what you mean by "discover file types".  There's no
discovery going on here.

> Of course, maybe that broad applicability of sniffing isn't appropriate, but 
> then ... where's are the boundaries?

The boundaries are exactly what's described in the document.  There's
been a great deal of research and implementation experience poured
into the document to determine precisely where to draw the boundaries.
 As far as I can tell, the document describes the optimal point.  If
you have data that shows otherwise, I'd like to see it.

> Which situations are in scope vs. not?

The criteria I would use is the following one:

"Given a diverse market of browser vendors, is this a sniffing
algorithm that all browser vendors are mutually interested in
converging upon."

If the answer is "yes", then you've identified the correct scope and
rules.  If "no", then the spec needs to be improved.  If there is no
such set of rules, then this endeavor is a waste of time and any spec
we create will be dead letter.

> And don't some of the "in-scope" situations need almost all MIME types to be 
> sniffable?

No.

Adam


> -Original Message-
> From: websec-boun...@ietf.org [mailto:websec-boun...@ietf.org] On Behalf Of 
> Adam Barth
> Sent: Tuesday, October 25, 2011 9:00 PM
> To: Tobias Gondrom
> Cc: websec@ietf.org
> Subject: Re: [websec] Issue 17: Registry for magic numbers
>
> Yeah, I think we're much better off creating a new registry rather than using 
> the MIME registry.  The truth is that most MIME types that get registered 
> don't need sniffing rules.  The only ones that need it are the legacy ones 
> and the ones browser vendor cause to need it because of the prisoner's 
> dilemma in the browser market.
>
> Adam
>
>
> On Tue, Oct 25, 2011 at 8:52 PM, Tobias Gondrom  
> wrote:
>> 
>> For me the point is, currently we have a table in the document, which
>> inside an RFC is rather static and hard to extend.
>> So it looks like a good case for a registry to allow for extendibility
>> for new mime-types. (e.g. we keep the table in the document, create an
>> IANA registry, copy the values to the registry and allow for future
>> entries by expert review) That can either be added to the current
>> Mime-type registry, or we create a new one (e.g. within the websec
>> namespace) with only these elements.
>>
>> Just my 5cents.
>>
>> Tobias
>>
>>
>>
>> On 25/10/11 05:23, Adam Barth wrote:
>>>
>>> On Mon, Oct 24, 2011 at 9:07 PM, "Martin J. Dürst"
>>>   wrote:

 On 2011/10/25 11:21, Adam Barth wrote:
>
> http://trac.tools.ietf.org/wg/websec/trac/ticket/17 refers to an
> IANA registry with magic numbers for various media types.  I wanted
> to compare them to what's in the draft, but I couldn't find it.  I
> found the media type registry, e.g., for images:
>
> http://www.iana.org/assignments/media-types/image/index.html
>
> but I don't see any magic numbers.  Would someone be willing to

Re: [websec] Issue 17: Registry for magic numbers

[Larry Masinter]

This gets back to the question of the scope of the document. Does it, or does 
it not, handle sniffing of arbitrary blobs of data that come in without any 
content-type, blobs of data labeled application/octet-stream, and data coming 
via ftp (through ftp URIs) or thumb drives or mounted NFS file systems or 
whatever? Does it, or does it not, handle sniffing rules inside ZIP packaged 
web applications?   

If it does, then sniffing should cover everything that is sniffable, including 
almost all MIME types  -- you say "most MIME types that get registered don't 
need sniffing rules", I don't know what the percentage is, but after all, don't 
you want to be able to discover file types??  


Of course, maybe that broad applicability of sniffing isn't appropriate, but 
then ... where's are the boundaries? Which situations are in scope vs. not? And 
don't some of the "in-scope" situations need almost all MIME types to be 
sniffable?

Larry


-Original Message-
From: websec-boun...@ietf.org [mailto:websec-boun...@ietf.org] On Behalf Of 
Adam Barth
Sent: Tuesday, October 25, 2011 9:00 PM
To: Tobias Gondrom
Cc: websec@ietf.org
Subject: Re: [websec] Issue 17: Registry for magic numbers

Yeah, I think we're much better off creating a new registry rather than using 
the MIME registry.  The truth is that most MIME types that get registered don't 
need sniffing rules.  The only ones that need it are the legacy ones and the 
ones browser vendor cause to need it because of the prisoner's dilemma in the 
browser market.

Adam


On Tue, Oct 25, 2011 at 8:52 PM, Tobias Gondrom  
wrote:
> 
> For me the point is, currently we have a table in the document, which 
> inside an RFC is rather static and hard to extend.
> So it looks like a good case for a registry to allow for extendibility 
> for new mime-types. (e.g. we keep the table in the document, create an 
> IANA registry, copy the values to the registry and allow for future 
> entries by expert review) That can either be added to the current 
> Mime-type registry, or we create a new one (e.g. within the websec 
> namespace) with only these elements.
>
> Just my 5cents.
>
> Tobias
>
>
>
> On 25/10/11 05:23, Adam Barth wrote:
>>
>> On Mon, Oct 24, 2011 at 9:07 PM, "Martin J. Dürst"
>>   wrote:
>>>
>>> On 2011/10/25 11:21, Adam Barth wrote:

 http://trac.tools.ietf.org/wg/websec/trac/ticket/17 refers to an 
 IANA registry with magic numbers for various media types.  I wanted 
 to compare them to what's in the draft, but I couldn't find it.  I 
 found the media type registry, e.g., for images:

 http://www.iana.org/assignments/media-types/image/index.html

 but I don't see any magic numbers.  Would someone be willing to 
 point me in the right direction?
>>>
>>> They are in the templates. To get the template for a registration, 
>>> start at the overview page 
>>> (http://www.iana.org/assignments/media-types/index.html).
>>>
>>> Then go to the page that lists all the registration for a give top 
>>> level, e.g. 
>>> http://www.iana.org/assignments/media-types/image/index.html for images.
>>>
>>> Then look at each registration template (click on the link in the 
>>> left column, or in the right column if the left one doesn't have a 
>>> link and the right one is to an RFC). You may then find a magic 
>>> number in the registration template. As an example, for image/jp2, 
>>> the template is at 
>>> http://www.iana.org/assignments/media-types/image/jp2.
>>>
>>> But it looks like earlier templates didn't have a field for a magic 
>>> number, and this and the reasons Anne gave make this information 
>>> helpful for cross-checking, but not much more.
>>
>> == Images ==
>>
>> PNG has a registration template
>> , but lacks a 
>> signature.
>> JPEG doesn't have a template.
>> GIF doesn't have a template.
>> BMP isn't even registered.
>> WEBP isn't even registered.
>> ICO has a registration template
>> > >
>> and has the correct signature.  Yay!
>>
>> == Text ==
>>
>> HTML lacks a registration template.
>>
>> == Application ==
>>
>> PDF doesn't have a template.
>> Postscript doesn't have a template.
>> OGG doesn't have a template.
>> RAR isn't even registered.
>> ZIP has a registration template
>> , but 
>> lacks a signature.
>> GZIP isn't even registered.
>> RSS isn't even registered.
>> Atom lacks a registration template.
>>
>> == Audio ==
>>
>> WAV isn't even registered.
>>
>> == Video ==
>>
>> MP4 lacks a registration template.
>> WebM isn't even registered.
>>
>> This does not look like a promising approach.  Note: I haven't even 
>> looked through all the registrations to see how many have signatures 
>> that we shouldn't be using.
>>
>> Adam
>> ___
>> websec mailing list
>> websec@ietf.org
>> https://www

Re: [websec] Issue 17: Registry for magic numbers

[Adam Barth]

Yeah, I think we're much better off creating a new registry rather
than using the MIME registry.  The truth is that most MIME types that
get registered don't need sniffing rules.  The only ones that need it
are the legacy ones and the ones browser vendor cause to need it
because of the prisoner's dilemma in the browser market.

Adam


On Tue, Oct 25, 2011 at 8:52 PM, Tobias Gondrom
 wrote:
> 
> For me the point is, currently we have a table in the document, which inside
> an RFC is rather static and hard to extend.
> So it looks like a good case for a registry to allow for extendibility for
> new mime-types. (e.g. we keep the table in the document, create an IANA
> registry, copy the values to the registry and allow for future entries by
> expert review)
> That can either be added to the current Mime-type registry, or we create a
> new one (e.g. within the websec namespace) with only these elements.
>
> Just my 5cents.
>
> Tobias
>
>
>
> On 25/10/11 05:23, Adam Barth wrote:
>>
>> On Mon, Oct 24, 2011 at 9:07 PM, "Martin J. Dürst"
>>   wrote:
>>>
>>> On 2011/10/25 11:21, Adam Barth wrote:

 http://trac.tools.ietf.org/wg/websec/trac/ticket/17 refers to an IANA
 registry with magic numbers for various media types.  I wanted to
 compare them to what's in the draft, but I couldn't find it.  I found
 the media type registry, e.g., for images:

 http://www.iana.org/assignments/media-types/image/index.html

 but I don't see any magic numbers.  Would someone be willing to point
 me in the right direction?
>>>
>>> They are in the templates. To get the template for a registration, start
>>> at
>>> the overview page
>>> (http://www.iana.org/assignments/media-types/index.html).
>>>
>>> Then go to the page that lists all the registration for a give top level,
>>> e.g. http://www.iana.org/assignments/media-types/image/index.html for
>>> images.
>>>
>>> Then look at each registration template (click on the link in the left
>>> column, or in the right column if the left one doesn't have a link and
>>> the
>>> right one is to an RFC). You may then find a magic number in the
>>> registration template. As an example, for image/jp2, the template is at
>>> http://www.iana.org/assignments/media-types/image/jp2.
>>>
>>> But it looks like earlier templates didn't have a field for a magic
>>> number,
>>> and this and the reasons Anne gave make this information helpful for
>>> cross-checking, but not much more.
>>
>> == Images ==
>>
>> PNG has a registration template
>> , but lacks a
>> signature.
>> JPEG doesn't have a template.
>> GIF doesn't have a template.
>> BMP isn't even registered.
>> WEBP isn't even registered.
>> ICO has a registration template
>> 
>> and has the correct signature.  Yay!
>>
>> == Text ==
>>
>> HTML lacks a registration template.
>>
>> == Application ==
>>
>> PDF doesn't have a template.
>> Postscript doesn't have a template.
>> OGG doesn't have a template.
>> RAR isn't even registered.
>> ZIP has a registration template
>> , but
>> lacks a signature.
>> GZIP isn't even registered.
>> RSS isn't even registered.
>> Atom lacks a registration template.
>>
>> == Audio ==
>>
>> WAV isn't even registered.
>>
>> == Video ==
>>
>> MP4 lacks a registration template.
>> WebM isn't even registered.
>>
>> This does not look like a promising approach.  Note: I haven't even
>> looked through all the registrations to see how many have signatures
>> that we shouldn't be using.
>>
>> Adam
>> ___
>> websec mailing list
>> websec@ietf.org
>> https://www.ietf.org/mailman/listinfo/websec
>
> ___
> websec mailing list
> websec@ietf.org
> https://www.ietf.org/mailman/listinfo/websec
>
___
websec mailing list
websec@ietf.org
https://www.ietf.org/mailman/listinfo/websec

Re: [websec] Issue 17: Registry for magic numbers

[Tobias Gondrom]

For me the point is, currently we have a table in the document, which 
inside an RFC is rather static and hard to extend.
So it looks like a good case for a registry to allow for extendibility 
for new mime-types. (e.g. we keep the table in the document, create an 
IANA registry, copy the values to the registry and allow for future 
entries by expert review)
That can either be added to the current Mime-type registry, or we create 
a new one (e.g. within the websec namespace) with only these elements.


Just my 5cents.

Tobias



On 25/10/11 05:23, Adam Barth wrote:

On Mon, Oct 24, 2011 at 9:07 PM, "Martin J. Dürst"
  wrote:

On 2011/10/25 11:21, Adam Barth wrote:

http://trac.tools.ietf.org/wg/websec/trac/ticket/17 refers to an IANA
registry with magic numbers for various media types.  I wanted to
compare them to what's in the draft, but I couldn't find it.  I found
the media type registry, e.g., for images:

http://www.iana.org/assignments/media-types/image/index.html

but I don't see any magic numbers.  Would someone be willing to point
me in the right direction?

They are in the templates. To get the template for a registration, start at
the overview page (http://www.iana.org/assignments/media-types/index.html).

Then go to the page that lists all the registration for a give top level,
e.g. http://www.iana.org/assignments/media-types/image/index.html for
images.

Then look at each registration template (click on the link in the left
column, or in the right column if the left one doesn't have a link and the
right one is to an RFC). You may then find a magic number in the
registration template. As an example, for image/jp2, the template is at
http://www.iana.org/assignments/media-types/image/jp2.

But it looks like earlier templates didn't have a field for a magic number,
and this and the reasons Anne gave make this information helpful for
cross-checking, but not much more.

== Images ==

PNG has a registration template
, but lacks a
signature.
JPEG doesn't have a template.
GIF doesn't have a template.
BMP isn't even registered.
WEBP isn't even registered.
ICO has a registration template

and has the correct signature.  Yay!

== Text ==

HTML lacks a registration template.

== Application ==

PDF doesn't have a template.
Postscript doesn't have a template.
OGG doesn't have a template.
RAR isn't even registered.
ZIP has a registration template
, but
lacks a signature.
GZIP isn't even registered.
RSS isn't even registered.
Atom lacks a registration template.

== Audio ==

WAV isn't even registered.

== Video ==

MP4 lacks a registration template.
WebM isn't even registered.

This does not look like a promising approach.  Note: I haven't even
looked through all the registrations to see how many have signatures
that we shouldn't be using.

Adam
___
websec mailing list
websec@ietf.org
https://www.ietf.org/mailman/listinfo/websec


___
websec mailing list
websec@ietf.org
https://www.ietf.org/mailman/listinfo/websec

[websec] #25: what, if any, sniffing for fonts is required?

[websec issue tracker]

#25: what, if any, sniffing for fonts is required?

 The current spec has a stub for sniffing fonts.
 The use case for this was @font-face, CSS' font linking feature.
 The request came in http://www.ietf.org/mail-
 archive/web/websec/current/msg00235.html

 However, "That seems very anecdotal.  Do you have data to back up these
 claims?" (in this case, "data" = "significant use cases where sniffing is
 necessary").


 http://lists.w3.org/Archives/Public/public-webfonts-wg/2011Apr/0005.html
 http://lists.w3.org/Archives/Public/public-webfonts-wg/2011Apr/0012.html

 Reading those, it looks like there was some disagreement about what types
 ought to be registered.  This seems like a case where there are multiple
 type definitions which can be distinguished by magic number or other usage
 patterns, and the question is whether to register them as separate types
 or to use a single type and disambiguate later in the process at the
 receiver.

 In any case, we need to resolve what font sniffing is necessary, what
 should be sniffed, etc.

-- 
+
 Reporter:  masinter@…  |  Owner:  draft-ietf-websec-mime-sniff@…
 Type:  defect  | Status:  new
 Priority:  major   |  Milestone:
Component:  mime-sniff  |Version:
 Severity:  -   |   Keywords:
+

Ticket URL: 
websec 

___
websec mailing list
websec@ietf.org
https://www.ietf.org/mailman/listinfo/websec