Re: [whatwg] Removing the need for separate feeds

[Ian Hickson]

On Fri, 22 May 2009, Dan Brickley wrote:
> On 22/5/09 09:21, Ian Hickson wrote:
> > On Fri, 22 May 2009, Henri Sivonen wrote:
> > > On May 22, 2009, at 09:01, Ian Hickson wrote:
> > > >USE CASE: Remove the need for feeds to restate the content of HTML
> > > > pages
> > > >(i.e. replace Atom with HTML).
> > > Did you do some kind of "Is this Good for the Web?" analysis on this
> > > one? That is, do things get better if there's yet another feed format?
> > 
> > As far as I can tell, things get better if the feed format and the default
> > output format are the same, yes. Generally, redundant information has
> > tended to lead to problems.
> 
> Would this include having a mechanism (microdata? xml islands?) that preserves
> extension markup from Atom feeds? eg. see
> http://www.ibm.com/developerworks/xml/library/x-extatom1/

Actually the algorithm to convert HTML to Atom doesn't even support all of 
Atom, let alone extensions. However, it's quite possible to extend HTML 
itself if it is to be used as a native feed format, as described here:

   
http://wiki.whatwg.org/wiki/FAQ#HTML5_should_support_a_way_for_anyone_to_invent_new_elements.21


On Fri, 22 May 2009, Adrian Sutton wrote:
> On 22/05/2009 08:21, "Ian Hickson"  wrote:
> > As far as I can tell, things get better if the feed format and the 
> > default output format are the same, yes. Generally, redundant 
> > information has tended to lead to problems.
> 
> Can you point to examples of this in relation to the use of feeds in 
> particular?

Smylers listed more than I could think of:

On Fri, 22 May 2009, Smylers wrote:
> 
> I can't find examples right now, but I have encountered various problems 
> along these lines in the past, including:
> 
> * The feed suddenly becomes empty.
> * A new blog has a 'feed' link, but it never works.
> * A blog's feed URL changes, but doesn't redirect.
> * A feed is misformatted in a way which causes it to be ignored.
> * The content of a feed is misformatted, such that in a feed reader its
>   display is mangled, such as HTML tags and entities showing, or spaces
>   having been squeezed out from around tags such that linked words don't
>   have spaces around them.
> * The content of a feed has certain critical information, such as an
>   image, stripped from it, such that it makes no sense, or has a
>   different meaning from the full post.
> * The content of a feed has certain critical mark-up stripped from it,
>   such as  around exponents in a mathematical expression rendering
>   "36" where "3 to the power of 6" was intended.
> 
> In all cases the HTML version of the blog had correctly displaying and 
> updating content; only the feed was affected by the issues.  This 
> usually left the author unaware of the problem, as they don't subscribe 
> to their own blog.


On Fri, 22 May 2009, Adrian Sutton wrote:
>
> This feels a lot like jumping the shark and solving a problem that has 
> already been solved at one end (syndicating content) and doesn't exist 
> at the other (syndicated content being out of sync with the HTML 
> version).

It seems like defining how one converts HTML to Atom is useful in general 
even if -- maybe even especially if -- the desire is to use Atom.


On Fri, 22 May 2009, Eduard Pascual wrote:
>
> While redundant *source* information easily leads to problems, for what 
> I have seen the sites using feeds tend to be almost always dynamic: both 
> the HTML pages and the feeds are generated via server scripts from the 
> *same set of source data*, normally from a database. This is especially 
> true for blogs, and any other CMS-based site, since CMSs normally rely a 
> lot on databases and server-side scripting. So on these cases we don't 
> actually have redundant information, but just multiple ways to retrieve 
> the same information.

That seems plausible, yes.


> For manually authored pages and feeds things would be different; but are 
> there really a significant ammount of such cases out there? I can't say 
> I have seen the entire web (who can?), but among what I have seen, I 
> have never encountered any hand authored feed, except for code examples 
> and similar "experimental" stuff.

On Fri, 22 May 2009, Toby Inkster wrote:
> 
> Surely this proves the need for a way of extracting feeds from HTML?

I don't know if it proves it per se, but it certainly indicates that there 
is a possible need.

I added the section on how to convert HTML pages to Atom based on requests 
over the years and most recently specifically in the context of the 
microdata section. It doesn't replace Atom, nor is anyone required to 
author HTML in any particular way because of this; it merely provides a 
migration path if one is desired. I think enabling this kind of 
interoperability between standards can only be good.


On Fri, 22 May 2009, Adrian Sutton wrote:
> On 22/05/2009 11:36, "Toby Inkster"  wrote:
> > 
> > You never see manually written feeds because people can't be bothered 
> > to manually wr

Re: [whatwg] Removing the need for separate feeds

[Eduard Pascual]

On 5/22/09, Eduard Pascual  wrote:
> [...]
> For manually authored pages and feeds things would be different; but
> are there really a significant ammount of such cases out there? I
> can't say I have seen the entire web (who can?), but among what I have
> seen, I have never encountered any hand authored feed, except for code
> examples and similar "experimental" stuff.

Please, let me clarify the intent of this comment was not to say "this
is pointless" nor anything like that; the goal was to encourage people
to post real-world examples of where this feature would be used so it
could be properly evaluated.

Now, having seen some of the cases, I must say that this addition
looks like a good idea, but it still needs some work (some issues and
shortcommings have already been highlighted).
There are cases where keeping a separate feed is still a good idea,
most prominently for site-wide feeds (because it's not possible to put
all the relevant stuff into a single HTML document, unless such
document is made for that purpose, but that would be a separate feed
on itself then), and for cases where the traffic on the feed is
significantly higher than for the document and/or the size of the
document is significantly bigger than the feed. These cases, however,
are just unaffected by the addition, and shouldn't prevent the
relveant ones to take benefit of it.

Regards,
Eduard Pascual

Re: [whatwg] Removing the need for separate feeds

[Kornel Lesinski]

On Fri, 22 May 2009 07:01:51 +0100, Ian Hickson  wrote:

> It doesn't collect the blogroll or the blog post tags yet, mostly because
> I'm not sure how to do that. Any suggestions of improvements are  
> naturally welcome.

There's hAtom that solves this problem already, and appears to have been 
proliferated by popular blogging software:
http://search.yahoo.com/search?p=searchmonkeyid%3Acom.yahoo.page.uf.hatom

but I doubt that many users take advantage of it. Almost all of these pages 
have standard feeds as well (and all of them can provide them via hAtom2Atom 
proxy).

Maybe a better approach would be to extend hAtom or define extraction in terms 
of hAtom? (e.g. make  and  interchangeable?)


> For each article element article that does not have an ancestor article 
> element

That excludes possibility of syndicating article's comments from markup like 
this:



post
comment
comment



Feed with only single entry "post" or "post comment comment" would not be 
useful.

OTOH it may be useful to include all nested comments in a single feed:

comment
comment reply



Another problem is that algorithm cannot create . Perhaps  
could be assumed if there's alternate link and article doesn't contain more 
than one header? Or has entire contents wrapped in ?


I haven't noticed any way to exclude articles from the feed (except hack 
...). I may have news that's not 
important enough to justify notification of all subscribers. Are trackbacks and 
tweets appropriate for ? I might want to show them on my page, but 
wouldn't want to repost them in my feeds.

-- 
regards, Kornel Lesinski

Re: [whatwg] Removing the need for separate feeds

[Kornel Lesinski]

On Fri, 22 May 2009 09:41:43 +0100, Eduard Pascual  wrote:

> For manually authored pages and feeds things would be different; but
> are there really a significant ammount of such cases out there? I
> can't say I have seen the entire web (who can?), but among what I have
> seen, I have never encountered any hand authored feed, except for code
> examples and similar "experimental" stuff.

I maintain hand-authored Atom feeds on few websites. It's not a problem if 
feeds are updated rarely. On websites without CMS copying&pasting few tags once 
a month feels like less work than moving website to a CMS or writing XSLT :)

Despite that I'm not excited about Ian's proposal. In these scenarios I often 
want content of the feed to be different than content of the page, e.g. feed 
says "I've added article about Foo.", but page has "Newest articles: Foo, Bar, 
Baz."

-- 
regards, Kornel Lesinski

Re: [whatwg] Removing the need for separate feeds

[Philip Taylor]

On Fri, May 22, 2009 at 2:02 PM, Adrian Sutton  wrote:
> On 22/05/2009 13:32, "Philip Taylor"  wrote:
>> Perhaps a page like http://philip.html5.org/data.html - people might
>> want to subscribe in their feed reader to see all the exciting
>> updates, and the markup is all hand-written. It's not at all like a
>> blog, but maybe it's data that could be usefully represented with
>> Atom.
>
> There are four articles on that page - do they really update often enough to
> warrant anything more than just adding plain If-Modified support to
> feedreaders and displaying the whole page when it changes?

The way I see it, there are 24 articles on the page (grouped into four
categories), each published independently at separate times. There
would be about a hundred if I kept that index up to date.

But I'm not sure this is a very compelling example, and I can't think
of any other cases where I'd possibly want to publish
non-database-backed data as both HTML and Atom.

-- 
Philip Taylor
exc...@gmail.com

Re: [whatwg] Removing the need for separate feeds

[Adrian Sutton]

On 22/05/2009 13:32, "Philip Taylor"  wrote:
> Perhaps a page like http://philip.html5.org/data.html - people might
> want to subscribe in their feed reader to see all the exciting
> updates, and the markup is all hand-written. It's not at all like a
> blog, but maybe it's data that could be usefully represented with
> Atom.

There are four articles on that page - do they really update often enough to
warrant anything more than just adding plain If-Modified support to
feedreaders and displaying the whole page when it changes?

Given the arguments for justifying the cost of additional attributes I've
seen go by on this list, this is probably the weakest I've seen and somehow
it made it into the draft.

HTML 5 doesn't need to solve every possible problem, nor should it try to.

Regards,

Adrian Sutton.
__
Adrian Sutton, CTO
UK: +44 1 753 27 2229  US: +1 (650) 292 9659 x717
Ephox 
Ephox Blogs , Personal Blog

Re: [whatwg] Removing the need for separate feeds

[Adrian Sutton]

On 22/05/2009 13:32, "Philip Taylor"  wrote:
> Perhaps a page like http://philip.html5.org/data.html - people might
> want to subscribe in their feed reader to see all the exciting
> updates, and the markup is all hand-written. It's not at all like a
> blog, but maybe it's data that could be usefully represented with
> Atom.

There are four articles on that page - do they really update often enough to
warrant anything more than just adding plain If-Modified support to
feedreaders and displaying the whole page when it changes?

Given the arguments for justifying the cost of additional attributes I've
seen go by on this list, this is probably the weakest I've seen and somehow
it made it into the draft.

HTML 5 doesn't need to solve every possible problem, nor should it try to.

Regards,

Adrian Sutton.
__
Adrian Sutton, CTO
UK: +44 1 753 27 2229  US: +1 (650) 292 9659 x717
Ephox 
Ephox Blogs , Personal Blog

Re: [whatwg] Removing the need for separate feeds

[Philip Taylor]

On Fri, May 22, 2009 at 11:45 AM, Adrian Sutton  wrote:
> [...]
> Can anyone point to examples where the content is entirely hand crafted and
> a feed would actually make sense?

Perhaps a page like http://philip.html5.org/data.html - people might
want to subscribe in their feed reader to see all the exciting
updates, and the markup is all hand-written. It's not at all like a
blog, but maybe it's data that could be usefully represented with
Atom.

Currently the markup looks like:

  
http://philip.html5.org/data/abbr-acronym.txt";>abbr,
acronym titles and contents. 
http://philip.html5.org/data/spaced-uris.txt";>URIs
containing spaces. 
...
  

If I understand the spec correctly, I would have to write something like:

  

  
http://philip.html5.org/data/abbr-acronym.txt";
rel="bookmark">abbr, acronym titles and
contents.
  

  
http://philip.html5.org/data/spaced-uris.txt";
rel="bookmark">URIs containing spaces.
  
...
  

and then it would hopefully work.

-- 
Philip Taylor
exc...@gmail.com

Re: [whatwg] Removing the need for separate feeds

[Dan Brickley]

On 22/5/09 12:36, Toby Inkster wrote:

Eduard Pascual wrote:


For manually authored pages and feeds things would be different; but
are there really a significant ammount of such cases out there? I
can't say I have seen the entire web (who can?), but among what I have
seen, I have never encountered any hand authored feed, except for code
examples and similar "experimental" stuff.


Surely this proves the need for a way of extracting feeds from HTML?

You never see manually written feeds because people can't be bothered to
manually write feeds. So the people who manually author HTML simply
don't bother providing feeds at all.

If an HTML page can *be* a feed, this allows manually authored HTML
pages to be subscribed to in feed readers.


FWIW the W3C homepage works this way since ~2000, 
http://www.w3.org/2000/08/w3c-synd/


cheers,

Dan

Re: [whatwg] Removing the need for separate feeds

[Smylers]

Adrian Sutton writes:

> On 22/05/2009 11:36, "Toby Inkster"  wrote:
> 
> > You never see manually written feeds because people can't be
> > bothered to manually write feeds. So the people who manually author
> > HTML simply don't bother providing feeds at all.
> > 
> > If an HTML page can *be* a feed, this allows manually authored HTML
> > pages to be subscribed to in feed readers.
> 
> For this to make sense, these people would also be manually adding new
> entries to the top of the page and dropping old ones off the bottom
> all by hand.  ...  Can anyone point to examples where the content is
> entirely hand crafted and a feed would actually make sense?

I find that 'news' pages on bands' and products' websites often seem to
be like this -- but I don't have links to any current ones (almost by
definition: because the sites don't offer feeds, I don't follow them, so
can't remember them).

Here are some from the past:

* http://warmscotland.org/

  News and updates were manually added to the top of the list.  The
  existed for a specific short-term campaign and I'm not sure any
  dropped off the bottom before it was over.

* http://web.archive.org/web/20051024131550/http://www.leeds-camra.com/Blog/

  Content was manually added to the top.  Once a month the content was
  moved to an archive page.  (This blog no longer exists.)

* http://web.archive.org/web/20060515220924/http://www.beccyowen.com/

  Content appeared at the top; I'm not sure if it ever disappeared from
  the bottom, until the entire site was redesigned.  (The redesign
  features a feed.)

* http://web.archive.org/web/20070827182615/www.arianeandzarina.com/blog.html

  Content was manually added at the top; I'm not sure if it ever
  disappeared from the bottom, until the entire blog was abandonned.
  The 'subscribe' link is to get e-mail updates.

Smylers

Re: [whatwg] Removing the need for separate feeds

[Brett Zamir]

I also wonder if feeds being accessible in HTML might give rise, as with 
stylesheets and scripts contained in the head (convenient as those can 
be too), to excessive bandwidth, as agents repeatedly request updates to 
a whole HTML page containing a lot of other data.


(If we had external entities working though, that might be different for 
XHTML at least, as the file could be included easily as well as reside 
in its own independently discoverable location (via )...)


Brett

Re: [whatwg] Removing the need for separate feeds

[Adrian Sutton]

On 22/05/2009 11:36, "Toby Inkster"  wrote:
> Surely this proves the need for a way of extracting feeds from HTML?
> 
> You never see manually written feeds because people can't be bothered to
> manually write feeds. So the people who manually author HTML simply
> don't bother providing feeds at all.
> 
> If an HTML page can *be* a feed, this allows manually authored HTML
> pages to be subscribed to in feed readers.

For this to make sense, these people would also be manually adding new
entries to the top of the page and dropping old ones off the bottom all by
hand.  Feeds aren't used for checking for updates to a page - they're used
to check for updates for a site (or section of a site). There are very few
cases where every item in a feed corresponds to the same page, even where
the entries may be aggregated into a single index page.

Even people who really want to edit pages in plain text editors tend to use
a system like Bloxsom to build those into an overall site, and generate the
feeds.

Can anyone point to examples where the content is entirely hand crafted and
a feed would actually make sense?

Regards,

Adrian Sutton.
__
Adrian Sutton, CTO
UK: +44 1 753 27 2229  US: +1 (650) 292 9659 x717
Ephox 
Ephox Blogs , Personal Blog

Re: [whatwg] Removing the need for separate feeds

[Toby Inkster]

Eduard Pascual wrote:

> For manually authored pages and feeds things would be different; but
> are there really a significant ammount of such cases out there? I
> can't say I have seen the entire web (who can?), but among what I have
> seen, I have never encountered any hand authored feed, except for code
> examples and similar "experimental" stuff.

Surely this proves the need for a way of extracting feeds from HTML?

You never see manually written feeds because people can't be bothered to
manually write feeds. So the people who manually author HTML simply
don't bother providing feeds at all.

If an HTML page can *be* a feed, this allows manually authored HTML
pages to be subscribed to in feed readers.

-- 
Toby Inkster 

Re: [whatwg] Removing the need for separate feeds

[Smylers]

Adrian Sutton writes:

> On 22/05/2009 08:21, "Ian Hickson"  wrote:
> 
> > As far as I can tell, things get better if the feed format and the
> > default output format are the same, yes. Generally, redundant
> > information has tended to lead to problems.
> 
> Can you point to examples of this in relation to the use of feeds in
> particular?

I can't find examples right now, but I have encountered various problems
along these lines in the past, including:

* The feed suddenly becomes empty.
* A new blog has a 'feed' link, but it never works.
* A blog's feed URL changes, but doesn't redirect.
* A feed is misformatted in a way which causes it to be ignored.
* The content of a feed is misformatted, such that in a feed reader its
  display is mangled, such as HTML tags and entities showing, or spaces
  having been squeezed out from around tags such that linked words don't
  have spaces around them.
* The content of a feed has certain critical information, such as an
  image, stripped from it, such that it makes no sense, or has a
  different meaning from the full post.
* The content of a feed has certain critical mark-up stripped from it,
  such as  around exponents in a mathematical expression rendering
  "36" where "3 to the power of 6" was intended.

In all cases the HTML version of the blog had correctly displaying and
updating content; only the feed was affected by the issues.  This
usually left the author unaware of the problem, as they don't subscribe
to their own blog.

Eduard Pascual writes:

> sites using feeds tend to be almost always dynamic: both the HTML
> pages and the feeds are generated via server scripts from the *same
> set of source data*,

I believe that to be true for at least most of the above cases I
encountered.  However that obviously wasn't sufficient to avoid the
problems.

> For manually authored pages and feeds things would be different; but
> are there really a significant ammount of such cases out there?

Not many.  But that's quite possibly because of the effort involved in
doing so.  The algorithm in the HTML 5 spec would allow some categories
of handcrafted pages to gain feeds for free.

I've often encountered webpages which I wished had feeds but don't.
It's possible that an algorithm such as this would encourage more pages
to do so.

Smylers

Re: [whatwg] Removing the need for separate feeds

[Eduard Pascual]

On Fri, May 22, 2009 at 9:21 AM, Ian Hickson  wrote:
> On Fri, 22 May 2009, Henri Sivonen wrote:
>> On May 22, 2009, at 09:01, Ian Hickson wrote:
>> >
>> >   USE CASE: Remove the need for feeds to restate the content of HTML pages
>> >   (i.e. replace Atom with HTML).
>>
>> Did you do some kind of "Is this Good for the Web?" analysis on this
>> one? That is, do things get better if there's yet another feed format?
>
> As far as I can tell, things get better if the feed format and the default
> output format are the same, yes. Generally, redundant information has
> tended to lead to problems.
IMO, feeds are the exception to confirm the rule. While redundant
*source* information easily leads to problems, for what I have seen
the sites using feeds tend to be almost always dynamic: both the HTML
pages and the feeds are generated via server scripts from the *same
set of source data*, normally from a database. This is especially true
for blogs, and any other CMS-based site, since CMSs normally rely a
lot on databases and server-side scripting. So on these cases we don't
actually have redundant information, but just multiple ways to
retrieve the same information.
For manually authored pages and feeds things would be different; but
are there really a significant ammount of such cases out there? I
can't say I have seen the entire web (who can?), but among what I have
seen, I have never encountered any hand authored feed, except for code
examples and similar "experimental" stuff.

Regards,
Eduard Pascual

Re: [whatwg] Removing the need for separate feeds

[Adrian Sutton]

On 22/05/2009 08:21, "Ian Hickson"  wrote:
> As far as I can tell, things get better if the feed format and the default
> output format are the same, yes. Generally, redundant information has
> tended to lead to problems.

Can you point to examples of this in relation to the use of feeds in
particular?

This feels a lot like jumping the shark and solving a problem that has
already been solved at one end (syndicating content) and doesn't exist at
the other (syndicated content being out of sync with the HTML version).

Regards,

Adrian Sutton
__
Adrian Sutton, CTO
UK: +44 1 753 27 2229  US: +1 (650) 292 9659 x717
Ephox 
Ephox Blogs , Personal Blog

Re: [whatwg] Removing the need for separate feeds

[Adrian Sutton]

On 22/05/2009 08:21, "Ian Hickson"  wrote:
> As far as I can tell, things get better if the feed format and the default
> output format are the same, yes. Generally, redundant information has
> tended to lead to problems.

Can you point to examples of this in relation to the use of feeds in
particular?

This feels a lot like jumping the shark and solving a problem that has
already been solved at one end (syndicating content) and doesn't exist at
the other (syndicated content being out of sync with the HTML version).

Regards,

Adrian Sutton
__
Adrian Sutton, CTO
UK: +44 1 753 27 2229  US: +1 (650) 292 9659 x717
Ephox 
Ephox Blogs , Personal Blog

Re: [whatwg] Removing the need for separate feeds

[Dan Brickley]

On 22/5/09 09:21, Ian Hickson wrote:

On Fri, 22 May 2009, Henri Sivonen wrote:

On May 22, 2009, at 09:01, Ian Hickson wrote:

   USE CASE: Remove the need for feeds to restate the content of HTML pages
   (i.e. replace Atom with HTML).

Did you do some kind of "Is this Good for the Web?" analysis on this
one? That is, do things get better if there's yet another feed format?


As far as I can tell, things get better if the feed format and the default
output format are the same, yes. Generally, redundant information has
tended to lead to problems.


Would this include having a mechanism (microdata? xml islands?) that 
preserves extension markup from Atom feeds? eg. see 
http://www.ibm.com/developerworks/xml/library/x-extatom1/


cheers,

Dan

Re: [whatwg] Removing the need for separate feeds

[Ian Hickson]

On Fri, 22 May 2009, Henri Sivonen wrote:
> On May 22, 2009, at 09:01, Ian Hickson wrote:
> > 
> >   USE CASE: Remove the need for feeds to restate the content of HTML pages
> >   (i.e. replace Atom with HTML).
> 
> Did you do some kind of "Is this Good for the Web?" analysis on this 
> one? That is, do things get better if there's yet another feed format?

As far as I can tell, things get better if the feed format and the default 
output format are the same, yes. Generally, redundant information has 
tended to lead to problems.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: [whatwg] Removing the need for separate feeds

[Henri Sivonen]

On May 22, 2009, at 09:01, Ian Hickson wrote:

  USE CASE: Remove the need for feeds to restate the content of HTML  
pages

  (i.e. replace Atom with HTML).



Did you do some kind of "Is this Good for the Web?" analysis on this  
one? That is, do things get better if there's yet another feed format?


--
Henri Sivonen
hsivo...@iki.fi
http://hsivonen.iki.fi/

[whatwg] Removing the need for separate feeds

[Ian Hickson]

One of the use cases I collected from the e-mails sent in over the past 
few months was the following:

   USE CASE: Remove the need for feeds to restate the content of HTML pages
   (i.e. replace Atom with HTML).

   SCENARIOS:
 * Paul maintains a blog and wishes to write his blog in such a way that
   tools can pick up his blog post tags, authors, titles, and his
   blogroll directly from his blog, so that he does not need to maintain
   a parallel version of his data in a "structured format." In other
   words, his HTML blog should be usable as its own structured feed.

   REQUIREMENTS:
 * Parsing rules should be unambiguous.
 * Should not require changes to HTML5 parsing rules.

I've attempted a first cut at the above here:

   http://www.whatwg.org/specs/web-apps/current-work/#atom

It doesn't collect the blogroll or the blog post tags yet, mostly because 
I'm not sure how to do that. Any suggestions of improvements are naturally 
welcome.

-- 
Ian Hickson   U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'