Re: Multidingus

[John MacFarlane]

+++ Waylan Limberg [Oct 21 12 12:49 ]:
>Very interesting examples in that FAQ.
> 
>Although I would say this one is a little misleading:
> 
>[1]http://johnmacfarlane.net/babelmark2/?text=%5C%5C%5Btest%5D(not+a+li
>nk%3F)
> 
>It would more accurately be this:
> 
>[2]http://johnmacfarlane.net/babelmark2/?text=%5C%5C%5Btest%5D(notalink
>%3F)
>While that is a lot less interesting, it is not exhibiting the issue of
>spaces in urls - which is the next item on your list.
> 
>There could also be this:
> 
>[3]http://johnmacfarlane.net/babelmark2/?text=%5C%5Btest%5D(notalink%3F
>)
> 
>Which is also not very interesting. The only difference in these two
>examples is the way php markdown handles escaped characters - which has
>nothing specific to do with links. Seems to me like none of
>these examples belong on the list.

Agreed. I was confused. I removed it.

John
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[Waylan Limberg]

Very interesting examples in that FAQ.

Although I would say this one is a little misleading:

http://johnmacfarlane.net/babelmark2/?text=%5C%5C%5Btest%5D(not+a+link%3F)

It would more accurately be this:

http://johnmacfarlane.net/babelmark2/?text=%5C%5C%5Btest%5D(notalink%3F)

While that is a lot less interesting, it is not exhibiting the issue of
spaces in urls - which is the next item on your list.

There could also be this:

http://johnmacfarlane.net/babelmark2/?text=%5C%5Btest%5D(notalink%3F)

Which is also not very interesting. The only difference in these two
examples is the way php markdown handles escaped characters - which has
nothing specific to do with links. Seems to me like none of
these examples belong on the list.

Waylan

On Sat, Oct 20, 2012 at 11:21 PM, John MacFarlane  wrote:

> Thanks!  I've added a FAQ:
>
> http://johnmacfarlane.net/babelmark2/faq.html
>
> This contains a longer list of interesting examples of differences
> between implementations, plus instructions on how to get your markdown
> implementation added to the comparisons.
>
> +++ David Chambers [Oct 19 12 15:16 ]:
> >This is a terrific tool, John. I now realize how much disagreement
> >there is between the different Markdown libs on certain topics (such
> as
> >consecutive lists).
> >
> >David
> >
> >On Friday, 19 October 2012 at 2:51 PM, John MacFarlane wrote:
> >
> >+++ Alan Hogan [Oct 19 12 14:07 ]:
> >
> >Here’s a tiny bookmarklet that can be run on Babelmark 2 as it stands
> >today, that replaces all the  elements on the page with an HTML
> >preview:
> >javascript:$('pre').each(function(i, el){ var html = $(el).text();
> >$(el).replaceWith($('').html(html)); });
> >Or, you may drag it into your bookmarks page from here:
> ><[1][1]http://peg.gd/2IU>
> >It’s not Good Code and doesn’t do anything useful like add tabs to
> >switch between representations, but hey, I spent five minutes on this
> >and it’s kind of useful, so I’m sharing it.
> >
> >Alan,
> >This is useful. I've incorporated it into the site itself, so you
> >can now toggle between "Preview" and "Code" modes. When I have a bit
> >of time I'll make it use tabs for this instead -- that would be nicer.
> >John
> >___
> >Markdown-Discuss mailing list
> >[2]Markdown-Discuss@six.pairlist.net
> >[3]http://six.pairlist.net/mailman/listinfo/markdown-discuss
> >
> > References
> >
> >1. http://peg.gd/2IU
> >2. mailto:Markdown-Discuss@six.pairlist.net
> >3. http://six.pairlist.net/mailman/listinfo/markdown-discuss
>
> > ___
> > Markdown-Discuss mailing list
> > Markdown-Discuss@six.pairlist.net
> > http://six.pairlist.net/mailman/listinfo/markdown-discuss
>
> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss
>



-- 

\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[John MacFarlane]

Thanks!  I've added a FAQ:

http://johnmacfarlane.net/babelmark2/faq.html

This contains a longer list of interesting examples of differences
between implementations, plus instructions on how to get your markdown
implementation added to the comparisons.

+++ David Chambers [Oct 19 12 15:16 ]:
>This is a terrific tool, John. I now realize how much disagreement
>there is between the different Markdown libs on certain topics (such as
>consecutive lists).
> 
>David
> 
>On Friday, 19 October 2012 at 2:51 PM, John MacFarlane wrote:
> 
>+++ Alan Hogan [Oct 19 12 14:07 ]:
> 
>Here’s a tiny bookmarklet that can be run on Babelmark 2 as it stands
>today, that replaces all the  elements on the page with an HTML
>preview:
>javascript:$('pre').each(function(i, el){ var html = $(el).text();
>$(el).replaceWith($('').html(html)); });
>Or, you may drag it into your bookmarks page from here:
><[1][1]http://peg.gd/2IU>
>It’s not Good Code and doesn’t do anything useful like add tabs to
>switch between representations, but hey, I spent five minutes on this
>and it’s kind of useful, so I’m sharing it.
> 
>Alan,
>This is useful. I've incorporated it into the site itself, so you
>can now toggle between "Preview" and "Code" modes. When I have a bit
>of time I'll make it use tabs for this instead -- that would be nicer.
>John
>___
>Markdown-Discuss mailing list
>[2]Markdown-Discuss@six.pairlist.net
>[3]http://six.pairlist.net/mailman/listinfo/markdown-discuss
> 
> References
> 
>1. http://peg.gd/2IU
>2. mailto:Markdown-Discuss@six.pairlist.net
>3. http://six.pairlist.net/mailman/listinfo/markdown-discuss

> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[David Chambers]

This is a terrific tool, John. I now realize how much disagreement there is 
between the different Markdown libs on certain topics (such as consecutive 
lists).  

David  


On Friday, 19 October 2012 at 2:51 PM, John MacFarlane wrote:

> +++ Alan Hogan [Oct 19 12 14:07 ]:
> > Here’s a tiny bookmarklet that can be run on Babelmark 2 as it stands
> > today, that replaces all the  elements on the page with an HTML
> > preview:
> >  
> > javascript:$('pre').each(function(i, el){ var html = $(el).text();
> > $(el).replaceWith($('').html(html)); });
> >  
> > Or, you may drag it into your bookmarks page from here:
> >  
> > <[1]http://peg.gd/2IU>
> >  
> > It’s not Good Code and doesn’t do anything useful like add tabs to
> > switch between representations, but hey, I spent five minutes on this
> > and it’s kind of useful, so I’m sharing it.
> >  
>  
>  
> Alan,
>  
> This is useful. I've incorporated it into the site itself, so you
> can now toggle between "Preview" and "Code" modes. When I have a bit
> of time I'll make it use tabs for this instead -- that would be nicer.
>  
> John
>  
>  
> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net (mailto:Markdown-Discuss@six.pairlist.net)
> http://six.pairlist.net/mailman/listinfo/markdown-discuss
>  
>  


___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[John MacFarlane]

+++ Alan Hogan [Oct 19 12 14:07 ]:
>Here’s a tiny bookmarklet that can be run on Babelmark 2 as it stands
>today, that replaces all the  elements on the page with an HTML
>preview:
> 
>javascript:$('pre').each(function(i, el){ var html = $(el).text();
>$(el).replaceWith($('').html(html)); });
> 
>Or, you may drag it into your bookmarks page from here:
> 
><[1]http://peg.gd/2IU>
> 
>It’s not Good Code and doesn’t do anything useful like add tabs to
>switch between representations, but hey, I spent five minutes on this
>and it’s kind of useful, so I’m sharing it.

Alan,

This is useful.  I've incorporated it into the site itself, so you
can now toggle between "Preview" and "Code" modes.  When I have a bit
of time I'll make it use tabs for this instead -- that would be nicer.

John


___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[Alan Hogan]

Here’s a tiny bookmarklet that can be run on Babelmark 2 as it stands today, 
that replaces all the  elements on the page with an HTML preview:

javascript:$('pre').each(function(i, el){ var html = $(el).text(); 
$(el).replaceWith($('').html(html)); });

Or, you may drag it into your bookmarks page from here:



It’s not Good Code and doesn’t do anything useful like add tabs to switch 
between representations, but hey, I spent five minutes on this and it’s kind of 
useful, so I’m sharing it.

AH

smime.p7s
Description: S/MIME cryptographic signature
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[Alan Hogan]

On Oct 19, 2012, at 1:53 PM, John MacFarlane  wrote:

> I've added peg-markdown, RedCarpet, and PythonMarkdown (thanks
> Waylan). I've also changed the display format so that implementations
> with identical output are consolidated.
> 
> Some fun examples to see how useful this can be:
> 
> http://johnmacfarlane.net/babelmark2/?text=%2B+++item+1%0D%0A%0D%0A%2B+++item+2%0D%0A%0D%0A+*+++*+++*+++*+++*


That’s really, really great, John.

Alan Hogan



smime.p7s
Description: S/MIME cryptographic signature
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[John MacFarlane]

I've added peg-markdown, RedCarpet, and PythonMarkdown (thanks
Waylan). I've also changed the display format so that implementations
with identical output are consolidated.

Some fun examples to see how useful this can be:

http://johnmacfarlane.net/babelmark2/?text=%2B+++item+1%0D%0A%0D%0A%2B+++item+2%0D%0A%0D%0A+*+++*+++*+++*+++*

http://johnmacfarlane.net/babelmark2/?text=x%3Cmax%3Ccode%3Ea%2Cb%3C/code%3E%0D%0A

http://johnmacfarlane.net/babelmark2/?text=***bold**+in+ital*%0A%0A***ital*+in+bold**

http://johnmacfarlane.net/babelmark2/?text=+8.+item+1%0A+9.+item+2%0A10.+item+2a

http://johnmacfarlane.net/babelmark2/?text=-+foo%0A-+bar%0A%0A1.+first%0A2.+second%0A

http://johnmacfarlane.net/babelmark2/?text=%5Bhi%5D+(%2Furl)%0A

http://johnmacfarlane.net/babelmark2/?text=%5Bhi%5D(%2Furl(with+parens))%0A

http://johnmacfarlane.net/babelmark2/?text=%5Bthis+%60is+a%5D+link%60+with+backticks%5D(%2Furl)%0A

http://johnmacfarlane.net/babelmark2/?text=%3CDIV%3E%0Ahi%0A%3C%2FDIV%3E%0A

http://johnmacfarlane.net/babelmark2/?text=%5BBei%C3%9F+nicht+in+die+Hand%2C+die+dich+f%C3%BCttert%5D%5B%5D%0A%0A%5BBEI%C3%9F+NICHT+IN+DIE+HAND%2C+DIE+DICH+F%C3%9CTTERT%5D%3A+http%3A%2F%2Fen.wikiquote.org%2Fwiki%2FGerman_proverbs%0A

http://johnmacfarlane.net/babelmark2/?text=1.+one%0A%0A2.+two%0A3.+three%0A

John

+++ John MacFarlane [Oct 19 12 11:01 ]:
> I've added Markdown 1.0.1, changed the title to BabelMark 2 (which is
> too good a name not to use), and moved the thing to
> 
> http://johnmacfarlane.net/babelmark2/
> 
> Also added a title tag.
> 
> So far we have pandoc, php markdown + extra, and both versions of
> Markdown.pl.  Anyone else want to implement a dingus server?
> I will probably make minor changes to the protocol in the next
> few days, but for now what you need is something that converts the
> contents of the 'text' parameter to HTML and returns a JSON object with
> the fields 'name', 'html', and 'version'.  Example:
> 
> $ curl 'http://johnmacfarlane.net/cgi-bin/pandoc-dingus?text=hi'
> {"name":"Pandoc","html":"hi","version":"1.9.4.2"}
> 
> +++ Michel Fortin [Oct 19 12 07:31 ]:
> > Le 2012-10-18 à 15:24, John MacFarlane  a écrit :
> > 
> > > I've implemented a version of what I described below.
> > > http://johnmacfarlane.net/pandoc/dingus.html
> > 
> > I see you've now added Markdown.pl 1.0.2b8, which is the latest non-public 
> > release made by John Gruber. Maybe you should add 1.0.1 too, like I've done 
> > for Babelmark, because it's the latest public release and it is probably 
> > the one in more widespread use. 1.0.2b8 and 1.0.1 do exhibit significant 
> > differences in some areas.
> > 
> > For instance, we were just talking about parens inside URLs and I just 
> > noticed that 1.0.2b8 does balance them correctly:
> > 
> > 
> > The HTML block parser was also completely redone in 1.0.2b4. Look at the 
> > changelog for for details (reproduced at the end of this email).
> > 
> > > Also, perhaps "dingus" isn't the best word for it, since it
> > > just displays the HTML source, not the formatted output.  I take
> > > it that's what we want, since this is intended primarily for
> > > comparing the output of different implementations on corner
> > > cases, not for users to get a feel for markdown.
> > 
> > 
> > If you want to call it Babelmark 2, you have my permission. I think it'll 
> > be a worthy successor. Also, you really ought to have a  tag on your 
> > page.
> > 
> > 
> > ## Changelog found in 1.0.2b8 ##
> > 
> > 1.0.2b8 - Wed 09 May 2007
> > 
> > +   Fixed bug with nested raw HTML tags that contained
> > attributes. The problem is that it uses a backreference in
> > the expression that it passes to gen_extract_tagged, which
> > is broken when Text::Balanced wraps it in parentheses.
> > 
> > Thanks to Matt Kraai for the patch.
> > 
> > +   Now supports URLs containing literal parentheses, such as:
> > 
> > http://en.wikipedia.org/wiki/WIMP_(computing)
> > 
> > Such parentheses may be arbitrarily nested, but must be
> > balanced.
> > 
> > 
> > 1.0.2b7
> > 
> > +   Changed shebang line from "/usr/bin/perl" to "/usr/bin/env perl"
> > 
> > +   Now only trim trailing newlines from code blocks, instead of 
> > trimming
> > all trailing whitespace characters.
> > 
> > 
> > 1.0.2b6 - Mon 03 Apr 2006
> > 
> > +   Fixed bad performance bug in new `Text::Balanced`-based 
> > block-level parser.
> > 
> > 
> > 1.0.2b5 - Thu 08 Dec 2005
> > 
> > +   Fixed bug where this:
> > 
> > [text](http://m.com "title" )
> > 
> > wasn't working as expected, because the parser wasn't allowing 
> > for spaces
> > before the closing paren.
> > 
> > 
> > 1.0.2b4 - Thu 08 Sep 2005
> >

Re: Multidingus

[John MacFarlane]

I've added Markdown 1.0.1, changed the title to BabelMark 2 (which is
too good a name not to use), and moved the thing to

http://johnmacfarlane.net/babelmark2/

Also added a title tag.

So far we have pandoc, php markdown + extra, and both versions of
Markdown.pl.  Anyone else want to implement a dingus server?
I will probably make minor changes to the protocol in the next
few days, but for now what you need is something that converts the
contents of the 'text' parameter to HTML and returns a JSON object with
the fields 'name', 'html', and 'version'.  Example:

$ curl 'http://johnmacfarlane.net/cgi-bin/pandoc-dingus?text=hi'
{"name":"Pandoc","html":"hi","version":"1.9.4.2"}

+++ Michel Fortin [Oct 19 12 07:31 ]:
> Le 2012-10-18 à 15:24, John MacFarlane  a écrit :
> 
> > I've implemented a version of what I described below.
> > http://johnmacfarlane.net/pandoc/dingus.html
> 
> I see you've now added Markdown.pl 1.0.2b8, which is the latest non-public 
> release made by John Gruber. Maybe you should add 1.0.1 too, like I've done 
> for Babelmark, because it's the latest public release and it is probably the 
> one in more widespread use. 1.0.2b8 and 1.0.1 do exhibit significant 
> differences in some areas.
> 
> For instance, we were just talking about parens inside URLs and I just 
> noticed that 1.0.2b8 does balance them correctly:
> 
> 
> The HTML block parser was also completely redone in 1.0.2b4. Look at the 
> changelog for for details (reproduced at the end of this email).
> 
> > Also, perhaps "dingus" isn't the best word for it, since it
> > just displays the HTML source, not the formatted output.  I take
> > it that's what we want, since this is intended primarily for
> > comparing the output of different implementations on corner
> > cases, not for users to get a feel for markdown.
> 
> 
> If you want to call it Babelmark 2, you have my permission. I think it'll be 
> a worthy successor. Also, you really ought to have a  tag on your page.
> 
> 
> ## Changelog found in 1.0.2b8 ##
> 
> 1.0.2b8 - Wed 09 May 2007
> 
>   +   Fixed bug with nested raw HTML tags that contained
>   attributes. The problem is that it uses a backreference in
>   the expression that it passes to gen_extract_tagged, which
>   is broken when Text::Balanced wraps it in parentheses.
> 
>   Thanks to Matt Kraai for the patch.
>   
>   +   Now supports URLs containing literal parentheses, such as:
>   
>   http://en.wikipedia.org/wiki/WIMP_(computing)
>   
>   Such parentheses may be arbitrarily nested, but must be
>   balanced.
> 
> 
> 1.0.2b7
> 
>   +   Changed shebang line from "/usr/bin/perl" to "/usr/bin/env perl"
>   
>   +   Now only trim trailing newlines from code blocks, instead of 
> trimming
>   all trailing whitespace characters.
> 
> 
> 1.0.2b6 - Mon 03 Apr 2006
> 
>   +   Fixed bad performance bug in new `Text::Balanced`-based 
> block-level parser.
> 
> 
> 1.0.2b5 - Thu 08 Dec 2005
> 
>   +   Fixed bug where this:
>   
>   [text](http://m.com "title" )
>   
>   wasn't working as expected, because the parser wasn't allowing 
> for spaces
>   before the closing paren.
> 
> 
> 1.0.2b4 - Thu 08 Sep 2005
> 
>   +   Filthy hack to support markdown='1' in div tags, because I need 
> it
>   to write today's fireball.
>   
>   +   First crack at a new, smarter, block-level HTML parser.
> 
> 1.0.2b3 - Thu 28 Apr 2005
> 
>   +   _DoAutoLinks() now supports the 'dict://' URL scheme.
> 
>   +   PHP- and ASP-style processor instructions are now protected as
>   raw HTML blocks.
> 
>   
>   <% ... %>
> 
>   +   Workarounds for regressions introduced with fix for "backticks 
> within
>   tags" bug in 1.0.2b1. The fix is to allow `...` to be turned 
> into
>   ... within an HTML tag attribute, and then to turn
>   these spurious `` tags back into literal backtick 
> characters
>   in _EscapeSpecialCharsWithinTagAttributes().
> 
>   The regression was caused because in the fix, we moved
>   _EscapeSpecialCharsWithinTagAttributes() ahead of _DoCodeSpans()
>   in _RunSpanGamut(), but that's no good. We need to process code
>   spans first, otherwise we can get tripped up by something like 
> this:
> 
>   ``
> 
> 
> 1.0.2b2 - 20 Mar 2005
> 
>   +   Fix for nested sub-lists in list-paragraph mode. Previously we 
> got
>   a spurious extra level of `` tags for something like this:
> 
>   *   this
> 
>   * 

Re: Multidingus

[Michel Fortin]

Le 2012-10-18 à 15:24, John MacFarlane  a écrit :

> I've implemented a version of what I described below.
> http://johnmacfarlane.net/pandoc/dingus.html

I see you've now added Markdown.pl 1.0.2b8, which is the latest non-public 
release made by John Gruber. Maybe you should add 1.0.1 too, like I've done for 
Babelmark, because it's the latest public release and it is probably the one in 
more widespread use. 1.0.2b8 and 1.0.1 do exhibit significant differences in 
some areas.

For instance, we were just talking about parens inside URLs and I just noticed 
that 1.0.2b8 does balance them correctly:


The HTML block parser was also completely redone in 1.0.2b4. Look at the 
changelog for for details (reproduced at the end of this email).

> Also, perhaps "dingus" isn't the best word for it, since it
> just displays the HTML source, not the formatted output.  I take
> it that's what we want, since this is intended primarily for
> comparing the output of different implementations on corner
> cases, not for users to get a feel for markdown.


If you want to call it Babelmark 2, you have my permission. I think it'll be a 
worthy successor. Also, you really ought to have a  tag on your page.


## Changelog found in 1.0.2b8 ##

1.0.2b8 - Wed 09 May 2007

+   Fixed bug with nested raw HTML tags that contained
attributes. The problem is that it uses a backreference in
the expression that it passes to gen_extract_tagged, which
is broken when Text::Balanced wraps it in parentheses.

Thanks to Matt Kraai for the patch.

+   Now supports URLs containing literal parentheses, such as:

http://en.wikipedia.org/wiki/WIMP_(computing)

Such parentheses may be arbitrarily nested, but must be
balanced.


1.0.2b7

+   Changed shebang line from "/usr/bin/perl" to "/usr/bin/env perl"

+   Now only trim trailing newlines from code blocks, instead of 
trimming
all trailing whitespace characters.


1.0.2b6 - Mon 03 Apr 2006

+   Fixed bad performance bug in new `Text::Balanced`-based 
block-level parser.


1.0.2b5 - Thu 08 Dec 2005

+   Fixed bug where this:

[text](http://m.com "title" )

wasn't working as expected, because the parser wasn't allowing 
for spaces
before the closing paren.


1.0.2b4 - Thu 08 Sep 2005

+   Filthy hack to support markdown='1' in div tags, because I need 
it
to write today's fireball.

+   First crack at a new, smarter, block-level HTML parser.

1.0.2b3 - Thu 28 Apr 2005

+   _DoAutoLinks() now supports the 'dict://' URL scheme.

+   PHP- and ASP-style processor instructions are now protected as
raw HTML blocks.


<% ... %>

+   Workarounds for regressions introduced with fix for "backticks 
within
tags" bug in 1.0.2b1. The fix is to allow `...` to be turned 
into
... within an HTML tag attribute, and then to turn
these spurious `` tags back into literal backtick 
characters
in _EscapeSpecialCharsWithinTagAttributes().

The regression was caused because in the fix, we moved
_EscapeSpecialCharsWithinTagAttributes() ahead of _DoCodeSpans()
in _RunSpanGamut(), but that's no good. We need to process code
spans first, otherwise we can get tripped up by something like 
this:

``


1.0.2b2 - 20 Mar 2005

+   Fix for nested sub-lists in list-paragraph mode. Previously we 
got
a spurious extra level of `` tags for something like this:

*   this

*   sub

that

+   Experimental support for [this] as a synonym for [this][].
(Note to self: No test yet for this.)
Be sure to test, e.g.: [permutations of this sort of [thing][].]


1.0.2b1 - 28  Feb 2005

+   Fix for backticks within HTML tag: like 
this

+   Fix for escaped backticks still triggering code spans:

There are two raw backticks here: \` and here: \`, not 
a code span

1.0.1 - 14 Dec 2004

1.0 - 28 Aug 2004

-- 
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca/

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[Alan Hogan]

On Oct 18, 2012, at 8:48 PM, John MacFarlane  wrote:

> I could store a secret token along with the URL of each dingus server.
> (It would be best to use a different token for each server.)  The
> multidingus would send this along with the request, and the servers
> could check for it.
> 
> I'm not sure how much additional security that really gives you,
> though.

Right. To be clear: No real additional *security* per se, but it does lighten 
the implementation burden for each individual dingus server: They don’t half to 
worry about throttling requests, checking length, etc., if the master 
multi-dingus server assumes those responsibilities.

And I think we can all agree that making it as simple as possible for someone 
to add a dingus server for their own implementation is a Good Thing.

> Right now I just use the jquery text() function to insert it into the "pre"; 
> this automatically escapes everything. 

100% sufficient and correct.

>>   3) If we show not just raw HTML but HTML previews as well, then yes, an
>>   XSS scrubber should be used. However, it isn’t probably huge deal
>>   if the multi-dingus (a) only accepts POST, not GET, requests for
>>   conversion, and (b) protects against CSRF attacks (many frameworks have
>>   built-in CSRF protection). Given those assumptions, essentially the
>>   only people who could suffer from bad output are the same people who
>>   gave us bad input.
> 
> I don't see any problems with using GET.  And it's certainly useful to be
> able to pass data to the script in the url; this way you can link to
> results in discussions, etc.

Absolutely, there are advantages to link-ability.

However, consider the case where all of these are true:

1) It’s possible, using merely GET, to control the input of the multi-dingus;
2) The multi-dingus displays an HTML preview, automatically or in a manner 
configurable via a GET param;
3) The HTML is _not_ sanitized or scrubbed

Then it is possible to send people a link that run arbitrary 
(attacker-controlled) javascript, on your server.

If there is no value to be gained by stealing a user session, and no way to 
automate the propagation of the link via JS on that page, then this still isn’t 
a huge deal. I may be being paranoid here; I spent a lot of time thinking about 
this sort of thing while building Blogic <http://themer.blogic.com/>.

Keep in mind that *not* scrubbing HTML makes for a slightly more complete 
service that can test more real-world edge cases. (Not everyone who uses 
Markdown accepts guest input or has a need or desire to run an XSS scrubber.)

I don’t want to decide as to which trade-offs to make, but I do want to make 
sure they are understood.

> I don't plan to display formatted HTML at all, so that cuts down one
> source of potential problems.

This crosses off item 2 in my list. However, it is probably self-evident that 
being able to see formatted / rendered HTML is a faster way to notice most 
inconsistencies than reading raw HTML. Is it bold in one parser and not in 
another? I can see that in a glance, much faster than checking which tags open 
and close where, especially when whitespace around tags is inconsistent between 
implementations.

My *own* gut says to make rendered HTML available in the multi-dingus, but only 
as a user-controlled option (e.g., a button that runs a script like 
`$('div').html($('pre').text())`, in jQuery terms). This also crosses off item 
2 in my list, without sacrificing scannability of output, if so desired.

> 
>>   4) In general, shouldn’t it be the multi-dingus’ job to protect against
>>   malicious code, instead of individual implementors? This would reduce
>>   the "attack surface."
> 
> I'm already doing everything I can think of in the multidingus.

Fantastic to hear.

Alan Hogan



smime.p7s
Description: S/MIME cryptographic signature
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[John MacFarlane]

+++ Alan Hogan [Oct 18 12 14:34 ]:
>On Oct 18, 2012, at 1:47 PM, Fletcher Penney
><[1]fletc...@fletcherpenney.net> wrote:
> 
>  Not to mention everyone will want to make sure to do some input
>  "sanitization" on the text input to try to filter mischievous input.
> 
>1) Consider *only* accepting requests from the multi-dingus. Perhaps it
>could be as simple as a shared secret: a GET param (called "token")
>with a value known only to the multi-dingus operator & the individual
>dingus hosts. (Checking HTTP REFERER wouldn’t allow the maintainer to
>test locally, and is easy to fake, anyway)

I could store a secret token along with the URL of each dingus server.
(It would be best to use a different token for each server.)  The
multidingus would send this along with the request, and the servers
could check for it.

I'm not sure how much additional security that really gives you,
though.

>2) If we are only showing raw HTML, then no one needs to worry about
>scrubbing against XSS/JS/HTML weirdness; merely the multi-dingus host
>must make sure to properly escape HTML when displaying the result.

Right.  The multidingus gets the data back as JSON.  Right now I just
use the jquery text() function to insert it into the "pre"; this
automatically escapes everything.  Other fields (name, author,
description, etc.) will also be treated as text and escaped.

>3) If we show not just raw HTML but HTML previews as well, then yes, an
>XSS scrubber should be used. However, it isn’t probably huge deal
>if the multi-dingus (a) only accepts POST, not GET, requests for
>conversion, and (b) protects against CSRF attacks (many frameworks have
>built-in CSRF protection). Given those assumptions, essentially the
>only people who could suffer from bad output are the same people who
>gave us bad input.

I don't see any problems with using GET.  And it's certainly useful to be
able to pass data to the script in the url; this way you can link to
results in discussions, etc.

Writers of individual dingus servers would still need to take steps to
avoid malicious attacks.  In general, this shouldn't be too hard.
Basically you'll be taking some text out of a json object,
running it through your markdown converter, and sending back the
result (together with some information about your implementation).

I don't plan to display formatted HTML at all, so that cuts down one
source of potential problems.

>4) In general, shouldn’t it be the multi-dingus’ job to protect against
>malicious code, instead of individual implementors? This would reduce
>the "attack surface."

I'm already doing everything I can think of in the multidingus.

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[Alan Hogan]

On Oct 18, 2012, at 1:47 PM, Fletcher Penney  
wrote:

> Not to mention everyone will want to make sure to do some input 
> "sanitization" on the text input to try to filter mischievous input.

1) Consider *only* accepting requests from the multi-dingus. Perhaps it could 
be as simple as a shared secret: a GET param (called "token") with a value 
known only to the multi-dingus operator & the individual dingus hosts. 
(Checking HTTP REFERER wouldn’t allow the maintainer to test locally, and is 
easy to fake, anyway)

2) If we are only showing raw HTML, then no one needs to worry about scrubbing 
against XSS/JS/HTML weirdness; merely the multi-dingus host must make sure to 
properly escape HTML when displaying the result.

3) If we show not just raw HTML but HTML previews as well, then yes, an XSS 
scrubber should be used. However, it isn’t probably huge deal if the 
multi-dingus (a) only accepts POST, not GET, requests for conversion, and (b) 
protects against CSRF attacks (many frameworks have built-in CSRF protection). 
Given those assumptions, essentially the only people who could suffer from bad 
output are the same people who gave us bad input.

4) In general, shouldn’t it be the multi-dingus’ job to protect against 
malicious code, instead of individual implementors? This would reduce the 
"attack surface."

Alan

smime.p7s
Description: S/MIME cryptographic signature
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[Fletcher Penney]

Would it be better to use a "POST" method than a "GET" method, and keep the raw 
text out of the URL?  It seems like that is begging for trouble  Of course, 
I haven't been working as much on the HTTP/CGI stuff lately and could be 
mistaken.

Not to mention everyone will want to make sure to do some input "sanitization" 
on the text input to try to filter mischievous input.


F-

On Oct 18, 2012, at 4:32 PM, John MacFarlane wrote:

> +++ John MacFarlane [Oct 18 12 15:24 ]:
> 
>> % curl 'http://johnmacfarlane.net/cgi-bin/pandoc-dingus?text=hi *there*'
>> {"html":"hi","version":"1.9.4.2"}
> 
> Sorry, that should be:
> 
> % curl
> % 'http://johnmacfarlane.net/cgi-bin/pandoc-dingus?text=hi+*there*'
> {"html":"hi there","version":"1.9.4.2"}%
> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss


-- 
Fletcher T. Penney
fletc...@fletcherpenney.net 



smime.p7s
Description: S/MIME cryptographic signature
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Multidingus

[John MacFarlane]

+++ John MacFarlane [Oct 18 12 15:24 ]:
 
> % curl 'http://johnmacfarlane.net/cgi-bin/pandoc-dingus?text=hi *there*'
> {"html":"hi","version":"1.9.4.2"}

Sorry, that should be:

% curl
% 'http://johnmacfarlane.net/cgi-bin/pandoc-dingus?text=hi+*there*'
{"html":"hi there","version":"1.9.4.2"}%
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Multidingus

[John MacFarlane]

I've implemented a version of what I described below.
http://johnmacfarlane.net/pandoc/dingus.html

So far the "multidingus" only supports pandoc, since there's
only a dingus server for pandoc.  But I'd like to add others
with your cooperation.

To implement a dingus server for your implementation, you
need to write a cgi script or server app that takes the
contents of the 'text' parameter, converts it to markdown,
and returns a json object with two properties:

  - version - a string with the version of your implementation
  - html - a string containing the html output for the input
   provided

Here's an example of my server in action:

% curl 'http://johnmacfarlane.net/cgi-bin/pandoc-dingus?text=hi *there*'
{"html":"hi","version":"1.9.4.2"}

If you send me the URL for your dingus server and the name of your
implementation, I'll add it to the "multidingus."  Other suggestions
also welcome.  For example, it might be useful to add fields to
the return value like "description", "website", and "author".

Also, perhaps "dingus" isn't the best word for it, since it
just displays the HTML source, not the formatted output.  I take
it that's what we want, since this is intended primarily for
comparing the output of different implementations on corner
cases, not for users to get a feel for markdown.

John

+++ John MacFarlane [Oct 18 12 09:40 ]:
> 
> Babelmark has very outdated versions of many implementations (e.g.
> pandoc 0.46, current version is 1.9.4.2.) And I don't blame the
> maintainer for not keeping up to date. It's a big job to keep up-to-date
> versions of umpteen implementations in many different languages going --
> especially while keeping an eye on security.
> 
> A while back on this list, I made the following suggestion.  Let's
> devise a protocol for a "dingus server" that each implementer can
> implement and keep up to date.  Each dingus server would receive text
> input as a POST request, or perhaps text plus some options, and return
> HTML output. (There could be a relatively short length limit if people
> are worried about users relying on the dingus server for regular text
> conversion.)
> 
> The central multidingus could then just be an HTML page with AJAX.
> It would take user input, then send out AJAX requests to all the dingus
> servers, consolidate the output, and display it.
> 
> When I, as a markdown implementer, update one of my implementations,
> I would just need to make sure I update the corresponding dingus server.
> That would be my responsibility, and the person who maintains the central
> multidingus needn't worry about it.  All the central multidingus needs are
> the URLs of all the dingus servers.
> 
> I think a multidingus like this would be *really* useful. What do people
> think?
> 
> John
> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss