Re: Pre-fetch rough draft

[Julian Reschke]

On 2012-11-08 17:31, Sergey Nikitin wrote:

...
Maybe you never visited the page or your cookies are expired (or you logged 
out).

And visiting any page (with all cookies/other headers) is irreversible action. 
It affects statistics, it could do something without your knowledge (marking 
message as read in your webmail?). Yes I know about special header browser 
should send to a server, but I've never heard about any site supporting it.
...


Visiting a page should be a "safe" action; we shouldn't design 
extensions to address cases where people get this wrong.



Also generating a whole page just for metadata in it is a waste of server's CPU 
time.


You don't need to generate the page (there's the HTTP HEAD method).

Best regards, Julian

Re: Pre-fetch rough draft

[Sergey Nikitin]

On 04.11.2012, at 2:10, Yehuda Katz wrote:

> Does this overlap with SPDY preloaded?
> 

I can't find anything about "SPDY preloaded".
Is it server push and server hint 
http://www.chromium.org/spdy/link-headers-and-server-hint
?

-- 
Sergey Nikitin

Re: Pre-fetch rough draft

[Sergey Nikitin]

On 06.11.2012, at 12:49, Julian Reschke wrote:

> On 2012-11-06 09:28, Sergey Nikitin wrote:
>> 
>> On 05.11.2012, at 16:28, Julian Reschke wrote:
>> 
 
 Yes. Exactly.
 It's not about offline apps, it's about reducing loading time.
>>> 
>>> There's already the "prefetch" link relation that you could use.
>>> 
>> 
>> You need at least two pages to start prefetching.
> 
> Why two?
> 
>> And you can't prefetch anything for the first page.
> 
> Yes, you can. Just use the first page's metadata instead of a separate 
> prefetch manifest,
> 

I mean If you already downloaded a page you don't need any metadata. You can 
parse it and extract all urls from script/style tags (Ok. Except for a 
dynamically inserted ones).

>> If you have single page application you can't prefetch.
> 
> Why?
> 
>> And it's not always possible for browser to visit a page (cookie/password 
>> protected).
> 
> It's always possible to "visit" the page; it just needs to return the 
> relevant header fields.
> 

Maybe you never visited the page or your cookies are expired (or you logged 
out).

And visiting any page (with all cookies/other headers) is irreversible action. 
It affects statistics, it could do something without your knowledge (marking 
message as read in your webmail?). Yes I know about special header browser 
should send to a server, but I've never heard about any site supporting it.

Also generating a whole page just for metadata in it is a waste of server's CPU 
time.
 
-- 
Sergey Nikitin

Re: Pre-fetch rough draft

[Julian Reschke]

On 2012-11-06 09:28, Sergey Nikitin wrote:


On 05.11.2012, at 16:28, Julian Reschke wrote:



Yes. Exactly.
It's not about offline apps, it's about reducing loading time.


There's already the "prefetch" link relation that you could use.



You need at least two pages to start prefetching.


Why two?


And you can't prefetch anything for the first page.


Yes, you can. Just use the first page's metadata instead of a separate 
prefetch manifest,



If you have single page application you can't prefetch.


Why?


And it's not always possible for browser to visit a page (cookie/password 
protected).


It's always possible to "visit" the page; it just needs to return the 
relevant header fields.



...


Best regards, Julian

Re: Pre-fetch rough draft

[Sergey Nikitin]

On 05.11.2012, at 16:28, Julian Reschke wrote:

>> 
>> Yes. Exactly.
>> It's not about offline apps, it's about reducing loading time.
> 
> There's already the "prefetch" link relation that you could use.
> 

You need at least two pages to start prefetching.
And you can't prefetch anything for the first page.
If you have single page application you can't prefetch.

And it's not always possible for browser to visit a page (cookie/password 
protected).
 

>> Prefetch manifest is a way to tell browser what should be downloaded in 
>> advance.
>> So when user opens the site (for the first time) all resources 
>> (css/js/images/...) are already cached.
>> 
>> And if later site's resources are updated browser could check prefest 
>> manifest and
>> re-download all new resources in background. Before user visited site again.
> 
> But then you don't need a manifest for that (see above).
> 
> Best regards, Julian
> 
> 

Re: Pre-fetch rough draft

[Charles McCathie Nevile]

On Tue, 30 Oct 2012 10:22:47 +0100, Charles McCathieNevile  
 wrote:



Hi,

I mentioned this and it's something we are working on.

Basic idea: site provides list of resources that it uses and can be  
cached for general improvements on the whole site. (We're seeing  
load-time improvement from 50% - 300% in our testing. We are using it on  
sites - mail.yandex.ru/prefetch.txt has an example).


mnot already suggested that we use http://tools.ietf.org/html/rfc5785 to  
locate our well-known URI, and we agree.


cheers

--
Charles McCathie Nevile - Consultant (web standards) CTO Office, Yandex
  cha...@yandex-team.ru Find more at http://yandex.com

Re: Pre-fetch rough draft

[Charles McCathie Nevile]

On Mon, 05 Nov 2012 13:28:43 +0100, Julian Reschke   
wrote:



On 2012-11-02 11:16, n...@yandex-team.ru wrote:

On Tue, 30 Oct 2012, Michael Nordman  wrote

The appcache is encumbered with guarantees about atomically updating a  
set of resources and then explicitly not hitting the network for them

...

This gist of this prefetch list seems different. More of a hint to warm
things up if possible and viola things are more responsive if those  
hints are taken.




Yes. Exactly.
It's not about offline apps, it's about reducing loading time.


There's already the "prefetch" link relation that you could use.


This makes sense if we agree that prefetching a manifest (see below for  
some reasons I think they serve different initial use cases) is the same  
as prefetching an individual resource. Personally I am not convinced that  
is true, although I don't know of any reason to feel strongly one way or  
the other.


Prefetch manifest is a way to tell browser what should be downloaded in  
advance. So when user opens the site (for the first time) all resources  
(css/js/images/...) are already cached.


And if later site's resources are updated browser could check prefest  
manifest and re-download all new resources in background. Before user

visited site again.


But then you don't need a manifest for that (see above).


I think you do in the case where the list of resources that are used by  
the site changes faster than the resources themselves.


e.g.
1. We use several versions of a script library, because different sites  
that rely on them update at different times. So we might point to v 1.1.1  
and v 1.1.2 and v 1.2.1 in different Yandex properties, updating the  
prefetch manifest while the cache-control says we're not changing the  
version-specific stuff.
2. We highlight some photos in a property. The photos are static resources  
- but the highlights change every so often (but not every page view or  
there would be no point in caching).


cheers

Chaals

--
Charles McCathie Nevile - Consultant (web standards) CTO Office, Yandex
  cha...@yandex-team.ru Find more at http://yandex.com

Re: Pre-fetch rough draft

[Julian Reschke]

On 2012-11-02 11:16, n...@yandex-team.ru wrote:

On Tue, 30 Oct 2012, Michael Nordman  wrote

The appcache is encumbered with guarantees about atomically updating a set
of resources and then explicitly not hitting the network for them once up
to date to ensure the site/app will function if the network really is
complete gone.

This gist of this prefetch list seems different. More of a hint to warm
things up if possible and viola things are more responsive if those hints
are taken.



Yes. Exactly.
It's not about offline apps, it's about reducing loading time.


There's already the "prefetch" link relation that you could use.


Prefetch manifest is a way to tell browser what should be downloaded in advance.
So when user opens the site (for the first time) all resources 
(css/js/images/...) are already cached.

And if later site's resources are updated browser could check prefest manifest 
and
re-download all new resources in background. Before user visited site again.


But then you don't need a manifest for that (see above).

Best regards, Julian

Re: Pre-fetch rough draft

[Yehuda Katz]

Does this overlap with SPDY preloaded?

-- Yehuda Katz
(ph) 718.877.1325
On Nov 3, 2012 3:35 PM,  wrote:

> > On Tue, 30 Oct 2012, Michael Nordman  wrote
> >
> > The appcache is encumbered with guarantees about atomically updating a
> set
> > of resources and then explicitly not hitting the network for them once up
> > to date to ensure the site/app will function if the network really is
> > complete gone.
> >
> > This gist of this prefetch list seems different. More of a hint to warm
> > things up if possible and viola things are more responsive if those hints
> > are taken.
> >
>
> Yes. Exactly.
> It's not about offline apps, it's about reducing loading time.
> Prefetch manifest is a way to tell browser what should be downloaded in
> advance.
> So when user opens the site (for the first time) all resources
> (css/js/images/...) are already cached.
>
> And if later site's resources are updated browser could check prefest
> manifest and
> re-download all new resources in background. Before user visited site
> again.
>
> --
> Sergey Nikitin
>
>
>

Re: Pre-fetch rough draft

[nop]

> On Tue, 30 Oct 2012, Michael Nordman  wrote
>
> The appcache is encumbered with guarantees about atomically updating a set
> of resources and then explicitly not hitting the network for them once up
> to date to ensure the site/app will function if the network really is
> complete gone.
>
> This gist of this prefetch list seems different. More of a hint to warm
> things up if possible and viola things are more responsive if those hints
> are taken.
>

Yes. Exactly.
It's not about offline apps, it's about reducing loading time.
Prefetch manifest is a way to tell browser what should be downloaded in advance.
So when user opens the site (for the first time) all resources 
(css/js/images/...) are already cached.

And if later site's resources are updated browser could check prefest manifest 
and
re-download all new resources in background. Before user visited site again.

-- 
Sergey Nikitin

Re: Pre-fetch rough draft

[Brady Eidson]

On Oct 30, 2012, at 11:19 AM, Julian Reschke  wrote:

> On 2012-10-30 10:57, Anne van Kesteren wrote:
>> On Tue, Oct 30, 2012 at 10:46 AM, Florian Bösch  wrote:
>>> The specification states that "Prefetch requests must not include
>>> cookies." which is not an effective measure to prevent user profiling.
>> 
>> I suspect it's to reduce the size of the request.
> 
> -> 
> 

Pre-fetch has a "must not" for omitting cookies, whereas browser hints are 
optional.

I tend to agree with this being a "must not" requirement.

~Brady

Re: Pre-fetch rough draft

[Michael Nordman]

The appcache is encumbered with guarantees about atomically updating a set
of resources and then explicitly not hitting the network for them once up
to date to ensure the site/app will function if the network really is
complete gone.

This gist of this prefetch list seems different. More of a hint to warm
things up if possible and viola things are more responsive if those hints
are taken.



On Tue, Oct 30, 2012 at 2:58 PM, Mounir Lamouri  wrote:

> On 10/30/2012 10:22 AM, Charles McCathieNevile wrote:
> > Hi,
> >
> > I mentioned this and it's somethign we are working on.
> >
> > Basic idea: site provides list of resources that it uses and can be
> > cached for general improvements on the whole site. (We're seeing
> > load-time improvement from 50% - 300% in our testing. We are using it on
> > sites - mail.yandex.ru/prefetch.txt has an example).
> >
> > The draft "spec" here is still very rough, but it shows what we've
> > implemented and some of what we think it is good for.
> >
> > This is meant as input to the appcache/packaging/etc discussion, and may
> > or may not be something this group takes on.
>
> At a first glance, it seems that appcache could do the same things.
> Which use cases appcache can't currently solve? In the process of fixing
> appcache, it could be interesting to add those use cases.
>
> --
> Mounir
>
>

Re: Pre-fetch rough draft

[Mounir Lamouri]

On 10/30/2012 10:22 AM, Charles McCathieNevile wrote:
> Hi,
> 
> I mentioned this and it's somethign we are working on.
> 
> Basic idea: site provides list of resources that it uses and can be
> cached for general improvements on the whole site. (We're seeing
> load-time improvement from 50% - 300% in our testing. We are using it on
> sites - mail.yandex.ru/prefetch.txt has an example).
> 
> The draft "spec" here is still very rough, but it shows what we've
> implemented and some of what we think it is good for.
> 
> This is meant as input to the appcache/packaging/etc discussion, and may
> or may not be something this group takes on.

At a first glance, it seems that appcache could do the same things.
Which use cases appcache can't currently solve? In the process of fixing
appcache, it could be interesting to add those use cases.

--
Mounir

Re: Pre-fetch rough draft

[Julian Reschke]

On 2012-10-30 11:44, Yuval Sadan wrote:

Adding another "well-known location" may cause (yet another) 404 error
for sites lacking this file, similar to favicon.ico. Plus it takes up
URL space.
I suggest that if such a file is used, it should not be in a "well-known
location" but a site specific location that can be specified by an HTTP
header and/or  tags.


That would be a Link header field, or a  tag.

Best regards, Julian

Re: Pre-fetch rough draft

[Julian Reschke]

On 2012-10-30 11:31, Brady Eidson wrote:


On Oct 30, 2012, at 11:19 AM, Julian Reschke  wrote:


On 2012-10-30 10:57, Anne van Kesteren wrote:

On Tue, Oct 30, 2012 at 10:46 AM, Florian Bösch  wrote:

The specification states that "Prefetch requests must not include
cookies." which is not an effective measure to prevent user profiling.


I suspect it's to reduce the size of the request.


-> 



Pre-fetch has a "must not" for omitting cookies, whereas browser hints are 
optional.

I tend to agree with this being a "must not" requirement.


That may be true, but still we shouldn't special-case certain URIs.

Best regards, Julian

Re: Pre-fetch rough draft

[Yuval Sadan]

Adding another "well-known location" may cause (yet another) 404 error for
sites lacking this file, similar to favicon.ico. Plus it takes up URL
space.
I suggest that if such a file is used, it should not be in a "well-known
location" but a site specific location that can be specified by an HTTP
header and/or  tags.

On Tue, Oct 30, 2012 at 11:22 AM, Charles McCathieNevile  wrote:

> Hi,
>
> I mentioned this and it's somethign we are working on.
>
> Basic idea: site provides list of resources that it uses and can be cached
> for general improvements on the whole site. (We're seeing load-time
> improvement from 50% - 300% in our testing. We are using it on sites -
> mail.yandex.ru/prefetch.txt has an example).
>
> The draft "spec" here is still very rough, but it shows what we've
> implemented and some of what we think it is good for.
>
> This is meant as input to the appcache/packaging/etc discussion, and may
> or may not be something this group takes on.
>
> cheers
>
> --
> Charles McCathie Nevile - private mail account

Re: Pre-fetch rough draft

[Julian Reschke]

On 2012-10-30 10:57, Anne van Kesteren wrote:

On Tue, Oct 30, 2012 at 10:46 AM, Florian Bösch  wrote:

The specification states that "Prefetch requests must not include
cookies." which is not an effective measure to prevent user profiling.


I suspect it's to reduce the size of the request.


-> 

Re: Pre-fetch rough draft

[Florian Bösch]

It's also a little unclear to me how prefetch improves the html
manifest or how they compare. It seems as if prefetch does pretty much
the same as the manifest except that it distributes fetching of the
actual resources out over time that the user might not be visiting a
site. Wouldn't it make more sense to tack this onto the html manifest?

On Tue, Oct 30, 2012 at 10:52 AM, Julian Reschke  wrote:
> On 2012-10-30 10:22, Charles McCathieNevile wrote:
>>
>> Hi,
>>
>> I mentioned this and it's somethign we are working on.
>>
>> Basic idea: site provides list of resources that it uses and can be
>> cached for general improvements on the whole site. (We're seeing
>> load-time improvement from 50% - 300% in our testing. We are using it on
>> sites - mail.yandex.ru/prefetch.txt has an example).
>>
>> The draft "spec" here is still very rough, but it shows what we've
>> implemented and some of what we think it is good for.
>>
>> This is meant as input to the appcache/packaging/etc discussion, and may
>> or may not be something this group takes on.
>>
>> cheers
>
>
> I'm not a caching expert, but adding yet another mechanism (in addition to
> HTTP caching and the HTML stuff) should be done with a lot of care.
>
> Nits: *if* you want to add yet another manifest file, please consider
> respecting .
>
> Also: I would try to remove hacks / special cases. If there's a need to
> declare resources that should be fetched without cookies, then we should
> have a generic mechanism for that, not something special-cased for a
> specific URI.
>
> Best regards, Julian
>
>
>

Re: Pre-fetch rough draft

[Julian Reschke]

On 2012-10-30 10:22, Charles McCathieNevile wrote:

Hi,

I mentioned this and it's somethign we are working on.

Basic idea: site provides list of resources that it uses and can be
cached for general improvements on the whole site. (We're seeing
load-time improvement from 50% - 300% in our testing. We are using it on
sites - mail.yandex.ru/prefetch.txt has an example).

The draft "spec" here is still very rough, but it shows what we've
implemented and some of what we think it is good for.

This is meant as input to the appcache/packaging/etc discussion, and may
or may not be something this group takes on.

cheers


I'm not a caching expert, but adding yet another mechanism (in addition 
to HTTP caching and the HTML stuff) should be done with a lot of care.


Nits: *if* you want to add yet another manifest file, please consider 
respecting .


Also: I would try to remove hacks / special cases. If there's a need to 
declare resources that should be fetched without cookies, then we should 
have a generic mechanism for that, not something special-cased for a 
specific URI.


Best regards, Julian

Re: Pre-fetch rough draft

[Anne van Kesteren]

On Tue, Oct 30, 2012 at 10:46 AM, Florian Bösch  wrote:
> The specification states that "Prefetch requests must not include
> cookies." which is not an effective measure to prevent user profiling.

I suspect it's to reduce the size of the request.


-- 
http://annevankesteren.nl/

Re: Pre-fetch rough draft

[Florian Bösch]

The specification states that "Prefetch requests must not include
cookies." which is not an effective measure to prevent user profiling.
For instance somebody could auto generate the prefetch.txt tailored to
the user to fetch URLs like to
http://somedomain.com/whatever?userID=1358f2d55b34fb581fd5479d079d91dd

I'd suggest to allow cookies since the no cookies restriction makes it
more difficult for non malicious uses, but does not effectively deter
malicious uses.

Since we've established that users can be uniquely identified by their
prefetch requests, there are some additional security concerns over
user profiling. For instance an ad-serving domain could generate user
tailored prefetch URLs for their ads. Traditionally ad-aggregators
could profile users browing behavior that way. However with this
additional tool they can also gauge the persistence of the domains
they served ads for in the users history. Additionally since some
browsers implement history/bookmark/tab syncing, it would now allow
those ad-providers to built a holistic picture of a user and all his
browsers/profiles across all his devices including all his history up
to the limit of history entries.


On Tue, Oct 30, 2012 at 10:22 AM, Charles McCathieNevile
 wrote:
>
> Hi,
>
> I mentioned this and it's somethign we are working on.
>
> Basic idea: site provides list of resources that it uses and can be cached 
> for general improvements on the whole site. (We're seeing load-time 
> improvement from 50% - 300% in our testing. We are using it on sites - 
> mail.yandex.ru/prefetch.txt has an example).
>
> The draft "spec" here is still very rough, but it shows what we've 
> implemented and some of what we think it is good for.
>
> This is meant as input to the appcache/packaging/etc discussion, and may or 
> may not be something this group takes on.
>
> cheers
>
> --
> Charles McCathie Nevile - private mail account