Re: Mysterious MD5ification under very specific circumstances
I happened across http://www.cforcoding.com/2010/01/markdown-musings-on-unintended.html as I was looking for information on why it was doing this. The comments are rather interesting: > Fred Blasdel said... > The PHP Markdown changelog should give you at least a hundred bugs in Markdown.pl to test against — he started with a straight transliteration (much like MarkdownSharp), and gradually made it less shitty. > Gruber's design 'escapes' blocks by replacing them with their hashcodes, but if the original input contains the same hashcodes — welcome to XSS city! Lesson learned: Just because markdown.pl is the implementation listed on the 'official' markdown page, and just because I can do `sudo apt-get install markdown` and get it, does *not* mean that it's the best implementation! I've switched to markdown_py (next one down on the list) and everything is working fine now. On Fri, Jun 28, 2013 at 12:44 PM, Fletcher Penney < fletc...@fletcherpenney.net> wrote: > By "markdown v.1.0.1" I'm guessing he meant Gruber's Perl Markdown.pl > 1.0.1. > > > As for whom to report the bug, Gruber's Markdown.pl is presumably not > going to be updated further, and has not been updated in years (also known > as an eternity in internet time... ;). Certainly someone on this list may > have an interest in finding the bug and posting a fix, but you may be > better off switching to a variant of Markdown that is still undergoing > active development. As Waylan hinted, there are lots, and I'm sure > everyone on the list has their favorites. Heck, many people on this list > have written their own (myself included). > > When choosing a variant, some things to consider: > > * What languages (if any) are you comfortable with if you want to change > anything? If you're not changing anything, this may not matter. > > * Are you using Markdown in a larger project where the choice of language > will have a significant impact on ease of use? > > * How important is performance --- there can be several orders of > magnitude difference between implementations? > > * Do you need extensions to the basic Markdown syntax (e.g. footnotes, > tables, etc.)? > I appreciate the suggestions; frankly, I'm just looking for something that will take basic markdown and HTMLify it so that I can make sure I got the syntax right. I'm actually embedding python unit tests in a markdown document via the doctest module, which is how I ran across this bug. I was trying to comment out some initialization that didn't need to be shown in the documentation, and suddenly my document went all funny. My own implementation, for example is [MultiMarkdown]( > http://fletcherpenney.net/multimarkdown/). It is written in C and is > designed to compile on pretty much anything. Once installed, you have a > simple binary that is extremely fast and easy to use, and offers a few > command line options. It's easily used in shell scripts, and most > languages offer the equivalent of a "system()" call so you can use external > utilities inside of Perl, ruby, etc. It offers a bunch of extra features > that many believe were missing from the original Markdown, but you can turn > those off with the compatibility mode to imitate the output from "standard > markdown", minus most of the bugs. ;) > > > Fletcher > > > > > On Jun 28, 2013, at 12:32 PM, Waylan Limberg wrote: > > > Wolfgang, > > > > Which implementation of the markdown parser are you using? Perl, php, > ruby, python, javascript, ... (and many more) and which version specificly? > > > > Ask to an explaination, some implementations of the parser use MD5 > Hashes as placeholders for the already parsed pieces of the document. My > guess is that you found an edge case which tripps up the code that swaps > out the placeholders for the parsed html. > > > > Waylan Limberg > > way...@gmail.com > > > > On Jun 28, 2013 11:55 AM, "Wolfgang Faust" > wrote: > > I was building a markdown document today when my document suddenly went > blank. When I looked at the HTML source, I found that all my codeblocks had > been MD5ified. The following is a minimal document which reproduces the > error: > > > > # Header # > > > > > > This is a codeblock. > > > > **Bold text** > > > > In particular, there must be: > > • A header > > • A comment containing the sequence NEWLINE TAB followed by at > least two greater-than signs > > • At least one codeblock > > • Bold text > > • Another comment at the end of the document. > > Changing even the smallest detail in the markdown results in a correct > HTML document, as expected. > > > > When I run this through markdown v.1.0.1, I get: > > Header > > > > > > > > What on earth is going on here, and who do I report this bug to? > Thank you, Fletcher and Waylan, for your help! ___ Markdown-Discuss mailing list Markdown-Discuss@six.pairlist.net http://six.pairlist.net/mailman/listinfo/markdown-discuss
Re: Mysterious MD5ification under very specific circumstances
By "markdown v.1.0.1" I'm guessing he meant Gruber's Perl Markdown.pl 1.0.1. As for whom to report the bug, Gruber's Markdown.pl is presumably not going to be updated further, and has not been updated in years (also known as an eternity in internet time... ;). Certainly someone on this list may have an interest in finding the bug and posting a fix, but you may be better off switching to a variant of Markdown that is still undergoing active development. As Waylan hinted, there are lots, and I'm sure everyone on the list has their favorites. Heck, many people on this list have written their own (myself included). When choosing a variant, some things to consider: * What languages (if any) are you comfortable with if you want to change anything? If you're not changing anything, this may not matter. * Are you using Markdown in a larger project where the choice of language will have a significant impact on ease of use? * How important is performance --- there can be several orders of magnitude difference between implementations? * Do you need extensions to the basic Markdown syntax (e.g. footnotes, tables, etc.)? My own implementation, for example is [MultiMarkdown](http://fletcherpenney.net/multimarkdown/). It is written in C and is designed to compile on pretty much anything. Once installed, you have a simple binary that is extremely fast and easy to use, and offers a few command line options. It's easily used in shell scripts, and most languages offer the equivalent of a "system()" call so you can use external utilities inside of Perl, ruby, etc. It offers a bunch of extra features that many believe were missing from the original Markdown, but you can turn those off with the compatibility mode to imitate the output from "standard markdown", minus most of the bugs. ;) Fletcher On Jun 28, 2013, at 12:32 PM, Waylan Limberg wrote: > Wolfgang, > > Which implementation of the markdown parser are you using? Perl, php, ruby, > python, javascript, ... (and many more) and which version specificly? > > Ask to an explaination, some implementations of the parser use MD5 Hashes as > placeholders for the already parsed pieces of the document. My guess is that > you found an edge case which tripps up the code that swaps out the > placeholders for the parsed html. > > Waylan Limberg > way...@gmail.com > > On Jun 28, 2013 11:55 AM, "Wolfgang Faust" wrote: > I was building a markdown document today when my document suddenly went > blank. When I looked at the HTML source, I found that all my codeblocks had > been MD5ified. The following is a minimal document which reproduces the error: > > # Header # > > > This is a codeblock. > > **Bold text** > > In particular, there must be: > • A header > • A comment containing the sequence NEWLINE TAB followed by at least > two greater-than signs > • At least one codeblock > • Bold text > • Another comment at the end of the document. > Changing even the smallest detail in the markdown results in a correct HTML > document, as expected. > > When I run this through markdown v.1.0.1, I get: > Header > > > > What on earth is going on here, and who do I report this bug to? > > ___ > Markdown-Discuss mailing list > Markdown-Discuss@six.pairlist.net > http://six.pairlist.net/mailman/listinfo/markdown-discuss > > ___ > Markdown-Discuss mailing list > Markdown-Discuss@six.pairlist.net > http://six.pairlist.net/mailman/listinfo/markdown-discuss -- Fletcher T. Penney fletc...@fletcherpenney.net smime.p7s Description: S/MIME cryptographic signature ___ Markdown-Discuss mailing list Markdown-Discuss@six.pairlist.net http://six.pairlist.net/mailman/listinfo/markdown-discuss
Re: Mysterious MD5ification under very specific circumstances
On Jun 28, 2013 12:32 PM, "Waylan Limberg" wrote: > > Wolfgang, > > Which implementation of the markdown parser are you using? Perl, php, ruby, python, javascript, ... (and many more) and which version specificly? Sorry I wasn't clearer (I mentioned it in passing, but it wasn't obvious): I'm using the perl markdown script from daringfireball.com, version 1.0.1. > Ask to an explaination, some implementations of the parser use MD5 Hashes as placeholders for the already parsed pieces of the document. My guess is that you found an edge case which tripps up the code that swaps out the placeholders for the parsed html. That was my assumption, but I wasn't sure if this was a known bug or not. > Waylan Limberg > way...@gmail.com > > On Jun 28, 2013 11:55 AM, "Wolfgang Faust" wrote: >> >> I was building a markdown document today when my document suddenly went blank. When I looked at the HTML source, I found that all my codeblocks had been MD5ified. The following is a minimal document which reproduces the error: >> >> # Header # >> >> >> This is a codeblock. >> >> **Bold text** >> >> In particular, there must be: >> A header >> A comment containing the sequence NEWLINE TAB followed by at least two greater-than signs >> At least one codeblock >> Bold text >> Another comment at the end of the document. >> Changing even the smallest detail in the markdown results in a correct HTML document, as expected. >> >> When I run this through markdown v.1.0.1, I get: >> Header >> >> >> >> What on earth is going on here, and who do I report this bug to? >> >> ___ >> Markdown-Discuss mailing list >> Markdown-Discuss@six.pairlist.net >> http://six.pairlist.net/mailman/listinfo/markdown-discuss >> > > ___ > Markdown-Discuss mailing list > Markdown-Discuss@six.pairlist.net > http://six.pairlist.net/mailman/listinfo/markdown-discuss > ___ Markdown-Discuss mailing list Markdown-Discuss@six.pairlist.net http://six.pairlist.net/mailman/listinfo/markdown-discuss
Re: Mysterious MD5ification under very specific circumstances
Wolfgang, Which implementation of the markdown parser are you using? Perl, php, ruby, python, javascript, ... (and many more) and which version specificly? Ask to an explaination, some implementations of the parser use MD5 Hashes as placeholders for the already parsed pieces of the document. My guess is that you found an edge case which tripps up the code that swaps out the placeholders for the parsed html. Waylan Limberg way...@gmail.com On Jun 28, 2013 11:55 AM, "Wolfgang Faust" wrote: > I was building a markdown document today when my document suddenly went > blank. When I looked at the HTML source, I found that all my codeblocks had > been MD5ified. The following is a minimal document which reproduces the > error: > > # Header # > > > This is a codeblock. > > **Bold text** > > In particular, there must be: > >- A header >- A comment containing the sequence NEWLINE TAB followed by at least >two greater-than signs >- At least one codeblock >- Bold text >- Another comment at the end of the document. > > Changing even the smallest detail in the markdown results in a correct > HTML document, as expected. > > When I run this through markdown v.1.0.1, I get: > Header > > > > What on earth is going on here, and who do I report this bug to? > > ___ > Markdown-Discuss mailing list > Markdown-Discuss@six.pairlist.net > http://six.pairlist.net/mailman/listinfo/markdown-discuss > > ___ Markdown-Discuss mailing list Markdown-Discuss@six.pairlist.net http://six.pairlist.net/mailman/listinfo/markdown-discuss
Mysterious MD5ification under very specific circumstances
I was building a markdown document today when my document suddenly went blank. When I looked at the HTML source, I found that all my codeblocks had been MD5ified. The following is a minimal document which reproduces the error: # Header # This is a codeblock. **Bold text** In particular, there must be: - A header - A comment containing the sequence NEWLINE TAB followed by at least two greater-than signs - At least one codeblock - Bold text - Another comment at the end of the document. Changing even the smallest detail in the markdown results in a correct HTML document, as expected. When I run this through markdown v.1.0.1, I get: Header What on earth is going on here, and who do I report this bug to? ___ Markdown-Discuss mailing list Markdown-Discuss@six.pairlist.net http://six.pairlist.net/mailman/listinfo/markdown-discuss
