Re: Agreeing on "Historical Markdown" (was: Re: text/markdown effort in IETF (invite))

[Waylan Limberg]

> On Jul 11, 2014, at 7:20 PM, John MacFarlane  wrote:
> 
> +++ Sean Leonard [Jul 11 14 01:54 ]:
> 
>> Since we cannot reach consensus on what ought to be "Standard Markdown" 
>> today, can the community reach consensus on "Historical Markdown"--of which 
>> I propose three working definitions?
> 
> I think the only sensible thing to refer to is John Gruber's Markdown
> syntax description, which is the canonical reference (even if it is very
> incomplete on the details).  Markdown.pl 1.0.1 and 1.0.2b7 are both
> buggy implementations.  Neither one is faithful to the syntax
> description.

I agree. There is one markdown -- the syntax rules. While there may be many 
implementations, they are all buggy -- whether intentional or not.

Actually, I might be persuaded that there there is two: the rules, and 
"extended markdown" -- which would be all intentional deviations from the 
rules. If your documents are "markdown" then they strictly follow the rules and 
mostly likely will be parsed by all markdown parsers the same way. However, if 
your document is "extended markdown", then all bets are off. Such a label is in 
effect saying; "Hey, this text document represents markdown text, but may not 
strictly be pure markdown text. Weird things may happen. Consider yourself 
warned." Beyond that, I see no need to specify anything further.

Waylan Limberg
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Agreeing on "Historical Markdown" (was: Re: text/markdown effort in IETF (invite))

[John MacFarlane]

+++ Sean Leonard [Jul 11 14 01:54 ]:

Since we cannot reach consensus on what ought to be "Standard 
Markdown" today, can the community reach consensus on "Historical 
Markdown"--of which I propose three working definitions?


I think the only sensible thing to refer to is John Gruber's Markdown
syntax description, which is the canonical reference (even if it is very
incomplete on the details).  Markdown.pl 1.0.1 and 1.0.2b7 are both
buggy implementations.  Neither one is faithful to the syntax
description.

To give just one example: the syntax description says,
"Each subsequent paragraph in a list item must be indented by either 4
spaces or one tab."  But neither version of Markdown.pl actually
imposes this requirement:

http://johnmacfarlane.net/babelmark2/?normalize=1&text=+-+item%0A%0A+more%0A%0A+-+new+item%0A



___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Agreeing on "Historical Markdown" (was: Re: text/markdown effort in IETF (invite))

[Michel Fortin]

Le 11-juil.-2014 à 4:54, Sean Leonard  a écrit :

> Since we cannot reach consensus on what ought to be "Standard Markdown" 
> today, can the community reach consensus on "Historical Markdown"--of which I 
> propose three working definitions?
> 
> * Classic Markdown: The Markdown syntax or Markdown.pl implementation, as 
> implemented by John Gruber, in 1.0.1, with all ambiguities, bugs, 
> frustrations, and contradictions. [In cases that the syntax and the tool 
> contradict, we come up with a way to resolve the contradictions.]
> 
> * Original Markdown: The Markdown syntax or Markdown.pl implementation, as 
> implemented by John Gruber, in 1.0.2b7, with as many of the ambiguities, 
> bugs, frustrations, and contradictions fixed as he actually fixed (or failed 
> to fix) them. Aka "Markdown Web Dingus".
> 
> * Idealized Markdown (aka Historical Standard Markdown): The Markdown that 
> everyone can agree is the way Markdown "should have been" back when there was 
> One True Markdown. Basically this is Original Markdown with its faults duly 
> recognized and corrected...many of these faults having been corrected in 
> practice in divergent implementations (Markdown Extra etc.) but never 
> officially recognized in Original Markdown.
> 
> 
> I cannot say which of these three is better...but by recognizing these three 
> as common points, we can then start to compare on the same page.

You might also call the first two "Markdown 1.0.1" and "Markdown 1.0.2b7" for 
simplicity's sake. As for the idealized version, that's what I call "Markdown" 
personally, or "plain Markdown" when I need to disambiguate.

Wasn't 1.0.2b8 the last one though? Why is the Dingus running 1.0.2b7? 
Babelmark 2 has 1.0.2b8.


-- 
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sean Leonard]

On 7/10/2014 8:49 AM, Waylan Limberg wrote:

On Jul 09, 2014, at 11:49 AM, Sean Leonard  wrote:


Hi markdown-discuss Folks:

I am working on a Markdown effort in the Internet Engineering Task
Force, to standardize on "text/markdown" as the Internet media type for
all variations of Markdown content. You can read my draft here:
.


My response below is lengthy but covers a number of different points 
including some raised later in the discussion by others.


Sean, have you reached out to Mr. Gruber specifically?


Yes, I e-mailed him...twice. We shall see if he responds.


Why do we need a Mime Type?


In my own selfish use case, I want to identify Markdown files in my 
software projects as text/markdown (or *something*)...not just text/plain.


As I think I said before, Markdown is now being stored and exchanged 
"as-is". It is those "as-is" cases where we want to identify Markdown as 
it is, rather than its incorrect approximations (text/plain) or solely 
in its output formats (text/html, etc.).



Encodings
--
[...]

However, if the markdown text is included in a JSON file (see my 
previous point above), then wouldn't the encoding be defined for the 
JSON file, not the markdown text specifically. The JSON parsing 
library would just spit out a Unicode string -- in which case, why do 
we need this?


It's a general property of text streams (and in particular, a general 
property of Internet media types under the text/ main type) that the 
streams have a designated encoding. Frequently, this encoding is 
implicit, much like you can assume that Windows software uses 
little-endian byte ordering, or that line breaks on *nix-derived 
operating systems are , not  or . The issue comes down to 
interchange. When a Windows machine exchanges text files or data with a 
Unix machine, how do you represent the concept of newlines in a common way?


To your specific point: if you encase Markdown content in JSON content 
as a JSON string, you can probably assume the Markdown content uses the 
Unicode character set. But you can have Markdown content as-is in its 
own file or other protocol element, where there are no such assumptions. 
For example, in your JSON example, the JSON content is likely in a file 
(or XMLHttpRequest) that needs to be explicitly or implicitly labeled 
with UTF-8 encoding.




Flavors
-
[...]


Now that I know more, I am seriously rethinking that part of the 
proposal. Thanks for the info!


-Sean

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Agreeing on "Historical Markdown" (was: Re: text/markdown effort in IETF (invite))

[Sean Leonard]

So this thread has a lot of content, and is leading me to revise the 
proposal a few different ways. Thanks everyone thus far; it has been 
very educational.


I would like to ask the community here with a basic question, so I can 
start to reason out from there.


It seems that there is a general consensus that Markdown is an 
open-ended informal family of syntaxes based on John Gruber's original 
work. Everything in some way traces back to the original Markdown.pl 
script and syntax specification circa 2004. If it does not or cannot 
trace back, it's not Markdown--it's something else (e.g., reStructuredText).


At the same time, the proliferation of variations, extensions, fixes, 
tweaks, and everything else has led to a staunch lack of consensus on 
what constitutes "Standard Markdown", i.e., the Markdown in 2014 that 
/everyone ought to follow/. This is in contrast to, say, XML or 
HTML--with XML there are very strict standards of what one ought to 
follow; with HTML it's much more open-ended but at least there is one 
organization (W3C) and one "living standard" (HTML5) where people can 
glom on their kitchen-sink proposals.


Since we cannot reach consensus on what ought to be "Standard Markdown" 
today, can the community reach consensus on "Historical Markdown"--of 
which I propose three working definitions?


* Classic Markdown: The Markdown syntax or Markdown.pl implementation, 
as implemented by John Gruber, in 1.0.1, with all ambiguities, bugs, 
frustrations, and contradictions. [In cases that the syntax and the tool 
contradict, we come up with a way to resolve the contradictions.]


* Original Markdown: The Markdown syntax or Markdown.pl implementation, 
as implemented by John Gruber, in 1.0.2b7, with as many of the 
ambiguities, bugs, frustrations, and contradictions fixed as he actually 
fixed (or failed to fix) them. Aka "Markdown Web Dingus".


* Idealized Markdown (aka Historical Standard Markdown): The Markdown 
that everyone can agree is the way Markdown "should have been" back when 
there was One True Markdown. Basically this is Original Markdown with 
its faults duly recognized and corrected...many of these faults having 
been corrected in practice in divergent implementations (Markdown Extra 
etc.) but never officially recognized in Original Markdown.



I cannot say which of these three is better...but by recognizing these 
three as common points, we can then start to compare on the same page.


-Sean

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sean Leonard]

On 7/10/2014 3:46 AM, Carl Jacobsen wrote:

On Jul 9, 2014, at 3:07 PM, Michel Fortin  wrote:

I sure wish things would be simpler. But as things are now, I have a hard time identifying what 
"flavor" could mean. Should "Markdown.pl-1.0.1" be a flavor on its own?

Perhaps it would be better to have *two* optional fields:
Thanks Carl. I am starting to toy with this idea...I was thinking of 
calling it "markwith" (for what you call "processor") and "deviations" 
or "variations" (for what you--and I--call "flavor"). More on this shortly.


-Sean
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sreeharsha Mudivarti]

On Fri, Jul 11, 2014 at 3:58 AM, John MacFarlane  :

>
>
> Here is some math: \\(e=mc^2\\).
>
>
> Here's my current list of extensions/variations (from the pandoc
> source code).  Of course, it's nowhere near exhaustive:
>
>  Ext_footnotes   -- ^ Pandoc/PHP/MMD style footnotes
>| Ext_inline_notes-- ^ Pandoc-style inline notes
>| Ext_pandoc_title_block  -- ^ Pandoc title block
>| Ext_yaml_metadata_block -- ^ YAML metadata block


As much as admirable pandoc is, testing that is difficult.

Given that features can nested, the combinations is easily greater
than the number of lines in

https://github.com/jgm/pandoc/blob/master/tests/Tests/Readers/Markdown.hs

Markdown is an over-loaded term. It is safe to say that,
there are at-least two useful Markdowns

1) Plain

Useful for,

* Non-technical publishing
* Simple comment systems

( Markdown.pl )

2) Complete

* DocBook / Latex style publishing
* Editors

( MultiMarkdown )

A third type of Markdown can be labelled "Proprietary Markdown",
of which Github Markdown is a prime example.

Markdown implementations can give pre-processors and post-processors,
which can implement "Proprietary Extensions".

I don't understand why different things have to be conflated together.

If the IETF draft is from the perspective of publishing 1) can be ignored.
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[John MacFarlane]

+++ Aristotle Pagaltzis [Jul 10 14 14:18 ]:

• No implementor is going to write a Markdown processor that is actually
 capable of dealing with this MIME type. (Except of course the author
 of floobity 1.2.3 itself, in this example, of course – which is not
 for any attempt at making it work in a generic way.)


Pandoc can already do this. Quick demo:

pandoc -s -f markdown_strict+pandoc_title_block+tex_math_dollars -t 
markdown_strict+mmd_title_block+tex_math_double_backslash


% Demo
% John
% July 1, 2014

Here is some math:  $e=mc^2$.

^D
author: John
date: July 1, 2014
title: Demo

Here is some math: \\(e=mc^2\\).


Here's my current list of extensions/variations (from the pandoc
source code).  Of course, it's nowhere near exhaustive:

 Ext_footnotes   -- ^ Pandoc/PHP/MMD style footnotes
   | Ext_inline_notes-- ^ Pandoc-style inline notes
   | Ext_pandoc_title_block  -- ^ Pandoc title block
   | Ext_yaml_metadata_block -- ^ YAML metadata block
   | Ext_mmd_title_block -- ^ Multimarkdown metadata block
   | Ext_table_captions  -- ^ Pandoc-style table captions
   | Ext_implicit_figures-- ^ A paragraph with just an image is a figure
   | Ext_simple_tables   -- ^ Pandoc-style simple tables
   | Ext_multiline_tables-- ^ Pandoc-style multiline tables
   | Ext_grid_tables -- ^ Grid tables (pandoc, reST)
   | Ext_pipe_tables -- ^ Pipe tables (as in PHP markdown extra)
   | Ext_citations   -- ^ Pandoc/citeproc citations
   | Ext_raw_tex -- ^ Allow raw TeX (other than math)
   | Ext_raw_html-- ^ Allow raw HTML
   | Ext_tex_math_dollars-- ^ TeX math between $..$ or $$..$$
   | Ext_tex_math_single_backslash  -- ^ TeX math btw \(..\) \[..\]
   | Ext_tex_math_double_backslash  -- ^ TeX math btw \\(..\\) \\[..\\]
   | Ext_latex_macros-- ^ Parse LaTeX macro definitions (for math only)
   | Ext_fenced_code_blocks  -- ^ Parse fenced code blocks
   | Ext_fenced_code_attributes  -- ^ Allow attributes on fenced code blocks
   | Ext_backtick_code_blocks-- ^ Github style ``` code blocks
   | Ext_inline_code_attributes  -- ^ Allow attributes on inline code
   | Ext_markdown_in_html_blocks -- ^ Interpret as markdown inside HTML blocks
   | Ext_markdown_attribute  -- ^ Interpret text inside HTML as markdown
 --   iff container has attribute 'markdown'
   | Ext_escaped_line_breaks -- ^ Treat a backslash at EOL as linebreak
   | Ext_link_attributes -- ^ MMD style reference link attributes
   | Ext_autolink_bare_uris  -- ^ Make all absolute URIs into links
   | Ext_fancy_lists -- ^ Enable fancy list numbers and delimiters
   | Ext_lists_without_preceding_blankline -- ^ Allow lists without preceding 
blank
   | Ext_startnum-- ^ Make start number of ordered list significant
   | Ext_definition_lists-- ^ Definition lists as in pandoc, mmd, php
   | Ext_example_lists   -- ^ Markdown-style numbered examples
   | Ext_all_symbols_escapable  -- ^ Make all non-alphanumerics escapable
   | Ext_intraword_underscores  -- ^ Treat underscore inside word as literal
   | Ext_blank_before_blockquote -- ^ Require blank line before a blockquote
   | Ext_blank_before_header -- ^ Require blank line before a header
   | Ext_strikeout   -- ^ Strikeout using ~~this~~ syntax
   | Ext_superscript -- ^ Superscript using ^this^ syntax
   | Ext_subscript   -- ^ Subscript using ~this~ syntax
   | Ext_hard_line_breaks-- ^ All newlines become hard line breaks
   | Ext_ignore_line_breaks  -- ^ Newlines in paragraphs are ignored
   | Ext_literate_haskell-- ^ Enable literate Haskell conventions
   | Ext_abbreviations   -- ^ PHP markdown extra abbreviation definitions
   | Ext_auto_identifiers-- ^ Automatic identifiers for headers
   | Ext_ascii_identifiers   -- ^ ascii-only identifiers for headers
   | Ext_header_attributes   -- ^ Explicit header attributes {#id .class k=v}
   | Ext_mmd_header_identifiers -- ^ Multimarkdown style header identifiers 
[myid]
   | Ext_implicit_header_references -- ^ Implicit reference links for headers
   | Ext_line_blocks -- ^ RST style line blocks


___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[John MacFarlane]

+++ Carl Jacobsen [Jul 10 14 03:46 ]:



On Jul 9, 2014, at 3:07 PM, Michel Fortin  wrote:

I sure wish things would be simpler. But as things are now, I have a hard time identifying what 
"flavor" could mean. Should "Markdown.pl-1.0.1" be a flavor on its own?


Perhaps it would be better to have *two* optional fields:

- A "processor" field, as in `processor="Markdown.pl-1.0.1"` or  
`processor="Pandoc-1.12.4.2-nostrict", which indicates that the sender was using (that program) to 
view the output and was satisfied with it. This could be included whenever known, and hopefully *not* relied 
upon by the recipient, but could provide useful clues in some cases where the recipient *has* to decide how 
to interpret something.

- A "flavor" field consisting of zero or more alphanumeric tokens, separated by "+" or 
"," or some such, declaring well-understood deviations from, or extensions to, the original standard. Not as 
a completely exhaustive list (you don't have to be able to indicate that your processor has, say, special syntax for 
animated gif backgrounds if no one else can use that anyway), but simply to promote better interoperability, to be 
interpreted as "the text contained herein supports basic markdown, plus *at least* X and Y and Z 
capabilities". Include some useful things like:

   - "nofill" (hard returns should be obeyed rather than joining lines - this would 
clarify that "two github flavors" situation),
   - "tables" (supports some agreed-upon "least common denominator" table 
syntax),
   - "footnotes" (similar but for footnotes),
   - "fencedcodeblocks" (you get the idea),
   - "titleblock" (text lines before first blank line are some sort of metadata 
and shouldn't be displayed to casual viewers),
   - "restrictunderscores" (mid-word underscores are not to be interpreted as 
starting/ending emphasized or strong text)


I already made an attempt in pandoc to factor out some of these
dimensions of variability (so that different markdown flavors can be
converted to each other).  You can specify, e.g.,
markdown+pipe_tables+footnotes-tex_math_dollars.

See http://johnmacfarlane.net/pandoc/README.html#pandocs-markdown for
a list.

However, there are hundreds of more dimensions on which markdown
implementations vary (I could go on and on).  A crude list of
extensions/variations might be helpful, but I don't think you
could get close to a complete list.

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Alan Hogan]

On Jul 10, 2014, at 8:49 AM, Waylan Limberg  wrote:

> In my experience, REST API's (for example) use JSON or XML which may contain 
> some Markdown text among other data. That other data may identify that the 
> text is "markdown", but the mime type for the file is JSON or XML (or at 
> least the appropriate mime type for that file type). 

I’m no [REST Police][1], but in my understanding, REST encourages the embracing 
of HTTP verbs to perform actions on hypertext objects. Consider an API call 
that updates the Markdown source of a blog post, for example. You are entirely 
correct that there is a strong chance that this API call would actually send an 
updated copy of a JSON object including fields such as “title”, “date”, “url”, 
and “body”, the last of which may implicitly or explicitly be Markdown data. 
(And the MIME type on that call would be application/json or whatever.) But 
perhaps the most RESTful way to do this would be to send a plain Markdown file 
(as text/markdown). (As far as the metadata goes, the server could extract the 
title from the markdown document itself (first level-one heading or first line 
of text, for example), set the date automatically, and so on.

That’s not to say that a JSON API for updating a blog post isn’t RESTful, but 
rather that the non-JSON, pure-Markdown API is where the new MIME type would be 
most needed.

[1]: https://twitter.com/RESTPOLICE

Alan

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Aristotle Pagaltzis]

* Fletcher T. Penney  [2014-07-10 18:30]:
> I suppose it's debatable as to how "non-local" that effect is
> considered,

I do consider that local. “Local-enough” at least. Tellingly though it’s
also the most common single reason for “how do I” user inquiries on this
list.

> A less debatable example is fenced code blocks

Yes, true.

The annoying thing is, while I was against them when the proposal was
brought up for Markdown in general, after spending some time using them
(by way of GitHub), I do find them convenient as a user. So now I have
a dilemma.

— • —

Still though – because fenced code blocks are an extension, they don’t
much change the principle of what I said: if you use a fenced code
block, then try to render the document using some other processor which
doesn’t support them, you get a garbled code block and then the rest of
the document after that looks fine.

Conversely a documents written for processors that do not support fenced
code blocks are rather unlikely to contain something that looks as if it
were – which would then lead to large-scale botch if you tried to render
them using a processor that supports fenced code blocks. (I feel this is
especially so for backtick fences. Tilda fences seem to have some remote
likelihood of being used as an innocuous part of a non-fenced-code-block
document.)

Regards,
-- 
Aristotle Pagaltzis // 
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Fletcher T. Penney]

This isn't entirely true.  For example, try to insert a list immediately 
preceding an indented (not fenced) code block, where the code block is 
*not* part of the list.  By doing so, you convert the code block into a 
paragraph that becomes part of the last list item.  Indenting with one 
more tab results in a code block, but that code block is still part of 
the list.  This seems to be one of the first "gotchas" that gets many 
new users to Markdown.


I suppose it's debatable as to how "non-local" that effect is 
considered, since the code block immediately follows the list.  But 
inserting one structure (the list) breaks something that previously 
worked (the code block).


A less debatable example is fenced code blocks (not part of Markdown per 
se, but part of many derivatives.)  I was reluctant to include fenced 
code blocks in MultiMarkdown for just this reason.  To my chagrin, after 
adding fenced code block support to MMD, I then realized just how much 
of a bad idea they are.  They are the one syntax element you can't 
identify by simply looking local to that portion of the text.  You have 
to start all the way at the beginning, which can be a headache if you 
have a novel contained in a single text file.



Consider the following excerpt from a document:

```

This is *text*

```

This is more *text*

```

Without knowing the entire document before the "snippet", one cannot 
determine which of those sentences is a code block, and which is a 
paragraph.  The first row of single quotes could be starting a new code 
block, or it could be closing a previous code block.  One accidental 
fence delimiter at the beginning of a document could alter the meaning 
of the entire thing.  It's basically an "even/odd" problem, and the only 
way to know if a particular fence is even or odd is to count all the 
preceding fences.


This may not really matter that much when a single person creates a 
single document to be converted once into a single web page.  But in 
more complicated real world use, this can be problematic.


I don't know that there is much to do about it at the moment, and I 
don't currently plan to yank fenced code blocks from MMD.  But I mention 
it so that it can be considered when proposing additional new features 
on top of the Markdown base set.



FTP


On 7/10/14, 9:30 AM, Aristotle Pagaltzis wrote:

I’m afraid you entirely misunderstood what I meant by non-local.

What I was referring to is, e.g. in HTML you can insert tags at the top
of a document such as `` or `` which then change the way the
entire remaining document is to be rendered. They affect the document
non-locally.

Markdown does not have such constructs. If you include a Markdown Extra
table in the document and you put that document through Markdown.pl, you
will get a garbled form of the source of the table syntax as output for
the table, but the misrendering is only local. The rest of the document
will be unaffected and will render correctly.


--
Fletcher T. Penney
fletc...@fletcherpenney.net
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Waylan Limberg]

On Jul 09, 2014, at 11:49 AM, Sean Leonard  wrote:

Hi markdown-discuss Folks:

I am working on a Markdown effort in the Internet Engineering Task 
Force, to standardize on "text/markdown" as the Internet media type for 
all variations of Markdown content. You can read my draft here: 
.


My response below is lengthy but covers a number of different points including 
some raised later in the discussion by others. 

Sean, have you reached out to Mr. Gruber specifically? I mention this because 
in the past I have CCed him directly on a response I sent to this list which 
prompted him to respond (admittedly that happened some years ago). I suspect he 
might be amenable to the general idea though. A search of the list archives 
turned up a previous discussion [1] where he indicated a willingness to put in 
some work to obtain a mime type for markdown. Of course, that was back when he 
was still actively involved. Your mileage may vary.

[1]: http://article.gmane.org/gmane.text.markdown.general/1179

In any event, I have some thoughts about your proposal. I like it for the most 
part. But a few comments on some specifics:

Why do we need a Mime Type?


First of all, when is this necessary? In order words, when is plain markdown being sent 
around such that it needs a mime type? In my experience, REST API's (for example) use 
JSON or XML which may contain some Markdown text among other data. That other data may 
identify that the text is "markdown", but the mime type for the file is JSON or 
XML (or at least the appropriate mime type for that file type). Or are you proposing that 
everyone standardize on a way to identify the markdown text within JSON and XML documents 
as Markdown text? What am I missing here?

Encodings
--

To shed a different light on the encoding issue, consider Python-Markdown 
(disclosure: I'm the primary developer). Just as in Python 3 (where all strings 
are Unicode), Python-Markdown only works with Unicode. You pass Unicode text 
in, and you get Unicode text out. It is up to the user of the library to 
concern themselves with encoding and decoding a file to/from a specific 
encoding. As Python provides the libraries to do that, it is not a big problem 
-- although for those used to working with byte strings it may be a little 
jarring (I'm seeing that reaction from people who are experimenting with 
Apple's new Swift Language -- which also supports Unicode only strings).

The point is, the Python-Markdown implementation has no use for the encoding 
(except for the included wrapping commandline script). Of course, the user 
(user of the library) will care about that and will need some way to identify 
the encoding before converting and passing the input on to the Python-Markdown 
library. So yes, encoding is very much a real, needed piece of meta-data.

However, if the markdown text is included in a JSON file (see my previous point 
above), then wouldn't the encoding be defined for the JSON file, not the 
markdown text specifically. The JSON parsing library would just spit out a 
Unicode string -- in which case, why do we need this?

Flavors
-

To me, "flavors" seems like a disaster waiting to happen. Sean, I realize you 
have specifically stated a lack of understanding here, so lets go back in time. The 
following may not be an all-inclusive (or in proper order of events) history of Markdown, 
but provides enough (I hope) to make a point.

Way back when, the "flavor" of markdown you used depended almost entirely on which language (Perl, 
PHP, Python...) you were using to code your project (blog, wiki, CMS, etc.). If you where using PHP, them 
your flavor was PHP Markdown... There was only one implementation per language and they (mostly?) agreed with 
each other. In that day "flavor" was completely pointless. I suspect a number of us resistant to 
the "flavors" part of your proposal are from that period in Markdown's history.

Of course, then Ruby came along. I don't remember which library was which, but when the first 
library came out, it was not very good (lots of bugs and slow). Then a second library came out 
which also wasn't very good, but in different ways (except for the slow part). Some people wrote 
their markdown documents with the bugs of the first implementation in mind, while other's wrote 
their documents with the second in mind. Then a few projects started offering users the option to 
pick which Ruby implementation of Markdown to use for each individual document - and 
"flavors" were born. Then other people started making ports of those projects to other 
languages and the "flavors" followed -- even though the other languages didn't really 
have any choices. As a reminder, Github came out of that Ruby culture, which might explain why 
Github-Flavored-Markdown ever existed in the first place (interesting side note: Gruber appears to 

Re: text/markdown effort in IETF (invite)

[Aristotle Pagaltzis]

* Sean Leonard  [2014-07-10 06:30]:
> On 7/9/2014 8:06 PM, Aristotle Pagaltzis wrote:
> > Yet guessing wrong is largely without consequence.
> >
> > There are really no syntax features that affect the document’s
> > rendering non-locally. If part of a document is written with
> > unsupported syntax, only that part will render incorrectly, but the
> > other parts will come out fine.
>
> There are two use cases that I am particularly interested in: #1 You
> put .md files in a project (readme.md, etc.). These .md files are then
> passed around among project users, which may include developers,
> copy-writers, copy-editors, etc. They need to be sure that the
> readme.md is treated in the same way, which ought to be communicated
> with the data. If one person edits the document in UTF-8 and commits
> and another person edits the document in ISO-8859-1, you're going to
> have problems.
>
> #2 You have some app (let's say some web forum for example, but it
> literally could be anything, an electronic health record, some
> national criminal records, whatever) and you export data from the app.
> Say to some structured data format like XML or a sqlite database. Part
> of data liberation or backup or whatever. You want to get whatever
> your users actually input into the fields--not the HTMLized versions.
> So you need to annotate the blobs of data as Markdown, since users
> like to upload various kinds of data (Word docs, JPEG images, MP4
> videos, bits of text like names of individuals, whatever).
>
> In both cases, rendering matters "non-locally".

I’m afraid you entirely misunderstood what I meant by non-local.

What I was referring to is, e.g. in HTML you can insert tags at the top
of a document such as `` or `` which then change the way the
entire remaining document is to be rendered. They affect the document
non-locally.

Markdown does not have such constructs. If you include a Markdown Extra
table in the document and you put that document through Markdown.pl, you
will get a garbled form of the source of the table syntax as output for
the table, but the misrendering is only local. The rest of the document
will be unaffected and will render correctly.

> As someone new to Markdown development, I really want to see some
> comprehensive references (since "authority" in Markdown-land is notably
> absent).

I’m afraid you will have to first find and then survey all of processors
yourself. The closest there is to central coordination is discussion on
this list, but it’s more of a users list that a lot of implementors seem
to shun (partly or fully) and others are unaware of.

> Besides, since Markdown is such a free-for-all, someone could easily
> write a Markdown processor that turns (!) into
> alert('hello!');.

Sure, someone could, but who would use it? There is no point in basing
any technical considerations on this.

> > So you will get a document that differs from the author’s intent in
> > some way. But it will be clear *where* the differences are and you
> > will still get all of the data in *some* form, quite possibly fully
> > intelligible if not pretty.
>
> For what we might call "sensible flavors" of Markdown, yes. But the
> author's intent may be poorly represented when processed through
> a tool that injects lolcat pictures every third word. Or, the author's
> intent may be very well-represented.

It makes no sense to me to consider obviously silly pseudo-flavours just
because anything can claim to implement Markdown. What author is going
to write a real document using such a processor, and what user is going
to try and read Markdown documents with it?

> The point is...we don't know what the author's intent is, /unless the
> author tells us/.

And he has: he said it’s Markdown. It may not be entirely clear which
flavour, but that alone is a lot more than nothing. Now he sure should
be able to explain himself more specifically than that, but the user is
not dependent on more detail to make reasonably much sense out of the
document.

> > Therefore the flavour parameter ought to be considered nothing more
> > than loosely informative, and the processor should just render the
> > document to the best of its ability regardless of the flavour
> > specified. It MAY use the parameter value to adapt to the document,
> > in RFC 2119 lingo, but ought not be bound by it.
>
> I would reword this:
>
> The flavor parameter informs recipients of the author's intent. The
> processor should just render the document to the best of its ability
> regardless of the flavor specified. It SHOULD use the parameter value
> to adapt to the document.

MAY, MUST or bust. SHOULD is almost automatically bad idea and should be
employed very sparingly (though should also not be shied away from when
warranted).

Note that RFC 2119 “SHOULD” is not the same as English “should”.

> I don't know what should happen if the flavor is absent. I am trying
> to understand. Let me put it this way: if you come across un-annotated
> Markdown in the 

Re: text/markdown effort in IETF (invite)

[Aristotle Pagaltzis]

* Carl Jacobsen  [2014-07-10 12:50]:
> So, you might have:  content-type="text/markdown" flavor="tables+titleblock" 
> processor="floobity-1.2.3"

That’s very nice, except it will never happen.

• No user is going to annotate their files in such a way that this MIME
  type would ever show up in the wild in such specificity, unless they
  use special software which automatically records the relevant metadata
  (rather than just a text editor + file system workflow).

• No implementor is going to write a Markdown processor that is actually
  capable of dealing with this MIME type. (Except of course the author
  of floobity 1.2.3 itself, in this example, of course – which is not
  for any attempt at making it work in a generic way.)

• If the flavours refer to existing syntax extensions and modifications,
  those are all only vaguely specified, and the implementations that
  offer them are not likely to be changed to follow a more rigorous spec
  of the respective syntax extension any time soon. So in practice these
  flavour specs are no more well-defined than saying the document is
  (some kind of) Markdown.

• Documents are generally written for a specific processor, not for some
  particular combination of syntax extensions in the abstract. There are
  a number of processors which implement half a dozen separate syntax
  extensions. Documents written for these processors notionally employ
  all of these syntax extensions. Will documents in e.g. GitHub-Flavored
  Markdown always have to list the entire enchilada in their MIME type
  (instead of just saying “GFM”)? Or do you force whatever part of the
  infrastructure picks the MIME type to parse the document (and contain
  knowledge of all possible syntax extensions) to figure out which of
  them are actually used?

• I won’t make a separate bullet point for the endless effort of keeping
  track of all the possible syntax extensions, which is going to be
  a cat herder job, because that should be an obvious problem. But what
  if someone wants to implement a new syntax extension – what process do
  they have to go through before they can assign a truthful MIME type to
  their documents?

Metadata is hard. Let’s go shopping.

Regards,
-- 
Aristotle Pagaltzis // 
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Michel Fortin]

Le 10-juil.-2014 à 1:04, John MacFarlane  a écrit :

> +++ Michel Fortin [Jul 09 14 18:07 ]:
> 
>> Fun fact: PHP Markdown is mostly encoding agnostic. It understands UTF-8 
>> sequences but any byte that is not a valid UTF-8 sequence is treated as a 
>> character in itself. It's only relevant when converting tabs into spaces 
>> however, and only if you have non-ASCII characters before the tab.
> 
> Small amendment: There are at least two places where the difference
> between utf-8 and latin1 matters:  tab expansion (as you note) and
> reference links, since these are stipulated to be case insensitive.
> (Case conversion is sensitive to the encoding.)

Like Markdown.pl, PHP Markdown will just treat non-ASCII characters in a 
case-sensitive way so in my case it doesn't matter.

Also, if you want to compare characters in a case-sensitive manner, the most 
correct way to do it is to use the Unicode Collation Algorithm, not case 
conversion to lower or uppercase, because some characters can't round-trip (see 
[german ß]). Then you'll notice that unfortunately Unicode collation is locale 
dependent (because equivalent characters aren't the same in all locales, see 
the [turkish ı]). And then you'll realize there's not correct way to do it 
universally.

 [GERMAN SS]: https://en.wikipedia.org/wiki/ß
 [TURKISH I]: https://en.wikipedia.org/wiki/Turkish_dotted_and_dotless_I

On Babelmark I see that cheapskate 0.1.0.1 understands the first link above -- 
good job! -- an no one understands the second one.

http://johnmacfarlane.net/babelmark2/?normalize=1&text=Also%2C+if+you+want+to+compare+characters+in+a+case-sensitive+manner%2C+the+most+correct+way+to+do+it+is+to+use+the+Unicode+Collation+Algorithm+--+not+case+conversion+to+lower+or+uppercase+--+because+some+characters+can't+round-trip+(see+%5Bgerman+ß%5D).+Then+you'll+notice+that+unfortunately+Unicode+collation+is+locale+dependent+(because+equivalent+characters+aren't+the+same+in+all+locales%2C+see+the+%5Bturkish+ı%5D).+And+then+you'll+realize+there's+not+really+a+correct+way+to+do+it.%0A%0A+%5BGERMAN+SS%5D%3A+https%3A%2F%2Fen.wikipedia.org%2Fwiki%2Fß%0A+%5BTURKISH+I%5D%3A+https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FTurkish_dotted_and_dotless_I%0A

-- 
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Michel Fortin]

Le 10-juil.-2014 à 5:00, Sean Leonard  a écrit :

> I haven't tried it yet, but I suspect PHP Markdown is mostly encoding 
> agnostic only for most encodings that preserve the US-ASCII range.

That's what I meant by "mostly encoding agnostic". It'll work with ASCII and 
most European encoding schemes because they are ASCII-compatible, but anything 
more fancy than that will have to be UTF-8.

-- 
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sreeharsha Mudivarti]

>
> So, you might have:  content-type="text/markdown"
> flavor="tables+titleblock" processor="floobity-1.2.3"
>
>
That looks fine, but it has too many Combinations.

I don't think Markdown should be standardised.

It is,

1) Incomplete as compared to asciidoc

2) Self-Contradicting

No general escaping

3) No formal grammar

It is typically implemented by regexes.
So unlike html, a bold phrase, can't span two lines.

In the meantime you can standardise asciidoc, multimarkdown, wikicreole
which are saner than the original markdown and they already have specs.

Makrdown dubs its formatting on email conventions.
That's not true, because email conventions are free-form ascii.
Consider this email thread, for example.

Cheers,
Harsha
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Carl Jacobsen]

> On Jul 9, 2014, at 3:07 PM, Michel Fortin  wrote:
> 
> I sure wish things would be simpler. But as things are now, I have a hard 
> time identifying what "flavor" could mean. Should "Markdown.pl-1.0.1" be a 
> flavor on its own?

Perhaps it would be better to have *two* optional fields:

- A "processor" field, as in `processor="Markdown.pl-1.0.1"` or  
`processor="Pandoc-1.12.4.2-nostrict", which indicates that the sender was 
using (that program) to view the output and was satisfied with it. This could 
be included whenever known, and hopefully *not* relied upon by the recipient, 
but could provide useful clues in some cases where the recipient *has* to 
decide how to interpret something.

- A "flavor" field consisting of zero or more alphanumeric tokens, separated by 
"+" or "," or some such, declaring well-understood deviations from, or 
extensions to, the original standard. Not as a completely exhaustive list (you 
don't have to be able to indicate that your processor has, say, special syntax 
for animated gif backgrounds if no one else can use that anyway), but simply to 
promote better interoperability, to be interpreted as "the text contained 
herein supports basic markdown, plus *at least* X and Y and Z capabilities". 
Include some useful things like:

- "nofill" (hard returns should be obeyed rather than joining lines - this 
would clarify that "two github flavors" situation),
- "tables" (supports some agreed-upon "least common denominator" table 
syntax),
- "footnotes" (similar but for footnotes),
- "fencedcodeblocks" (you get the idea),
- "titleblock" (text lines before first blank line are some sort of 
metadata and shouldn't be displayed to casual viewers),
- "restrictunderscores" (mid-word underscores are not to be interpreted as 
starting/ending emphasized or strong text)

The idea being, merely having the "text/markdown" content type would not 
guarantee anything beyond "well, it's *some* sort of markdown flavored text", 
while specifying one or more of the attributes in the "flavor" field would 
indicate "yes, you can assume that if something looks like an X, it *is* an X" 
(where X is something from the list above). It doesn't necessarily mean you 
*will* find, say, a table, in the text, merely that if you see something that 
looks like a table, it likely is.

This way, senders can give a series of hints to make their text more clear to 
parse, using the "flavor" attribute, and can also declare what processor they 
were using to view/validate things on their end, in case the recipient feels 
they simply must know precisely how to interpret the text (at which point it's 
up to them to decide what, if any, conversion or special handling is 
necessary). And a recipient wouldn't need to consult an exhaustive list of 
processors, but simply look at the flavors attribute to see if it recognizes 
anything there as a useful hint (and render everything else naively).

So, you might have:  content-type="text/markdown" flavor="tables+titleblock" 
processor="floobity-1.2.3"

We'd need an agreed-upon list of, "ingredients", to put in the "flavors" field, 
but, again, you wouldn't want to exhaustively list every variant of every 
extension that any markdown-ish processor has ever come up with, only perhaps a 
dozen cases to cover the most common extensions (if you need to be more 
exacting, go consult the processor field and take matters into your own hands).

Does this make sense to anyone else? (I probably should have gone to sleep a 
long time ago.)

Cheers,
Carl Jacobsen
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sean Leonard]

On 7/9/2014 10:04 PM, John MacFarlane wrote:

+++ Michel Fortin [Jul 09 14 18:07 ]:

Fun fact: PHP Markdown is mostly encoding agnostic. It understands 
UTF-8 sequences but any byte that is not a valid UTF-8 sequence is 
treated as a character in itself. It's only relevant when converting 
tabs into spaces however, and only if you have non-ASCII characters 
before the tab.


Small amendment: There are at least two places where the difference
between utf-8 and latin1 matters:  tab expansion (as you note) and
reference links, since these are stipulated to be case insensitive.
(Case conversion is sensitive to the encoding.)


I haven't tried it yet, but I suspect PHP Markdown is mostly encoding 
agnostic only for most encodings that preserve the US-ASCII range. Try 
feeding it an EBCDIC-encoded file. The 0x20-0x3F codes in EBCDIC are not 
even printable characters! :)


And speaking of UTF-8, fun fact: there is a UTF-EBCDIC encoding that 
represents the whole Unicode repetoire in EBCDIC. See 
 and UTR #16.


-Sean


___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[John MacFarlane]

This seems a reasonable proposal to me.  Like Michel Fortin, though,
I suspect the "flavors" part will be more trouble than it's worth.
Is there going to be a distinct flavor for every version of pandoc,
for example?  What about people who use pandoc but disable one or
two of the pandoc extensions (which you can do with a command line
flag)?  Your document mentions "github flavored markdown," but there
are actually two distinct github flavors, one for displaying long-form
documents like READMEs (here hard line breaks are treated as spaces,
as in original markdown), and one of issues and comments (here hard
line breaks are rendered as hard breaks).  It sounds like a LOT of
work to keep the registry of flavors up to date, unless the flavors
are going to be very coarse-grained.

John

+++ Sean Leonard [Jul 09 14 08:49 ]:

Hi markdown-discuss Folks:

I am working on a Markdown effort in the Internet Engineering Task 
Force, to standardize on "text/markdown" as the Internet media type 
for all variations of Markdown content. You can read my draft here: .


The proposal is already getting traction. Is there anyone on this list 
that is interested in participating or helping this effort? In 
particular we need to better understand and document what versions of 
Markdown exist, so that either Markdown as a family of informal 
syntaxes will start to converge, or if not, that Markdown variations 
have an easy way to be distinguished from one another. (See the 
"flavor" parameter discussed in the draft.)


The draft is currently being discussed on apps-disc...@ietf.org.

Kind regards,

Sean Leonard
Author of Markdown IETF Draft
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[John MacFarlane]

+++ Michel Fortin [Jul 09 14 18:07 ]:


Fun fact: PHP Markdown is mostly encoding agnostic. It understands UTF-8 
sequences but any byte that is not a valid UTF-8 sequence is treated as a 
character in itself. It's only relevant when converting tabs into spaces 
however, and only if you have non-ASCII characters before the tab.


Small amendment: There are at least two places where the difference
between utf-8 and latin1 matters:  tab expansion (as you note) and
reference links, since these are stipulated to be case insensitive.
(Case conversion is sensitive to the encoding.)

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sean Leonard]

On 7/9/2014 8:06 PM, Aristotle Pagaltzis wrote:

* Sean Leonard  [2014-07-09 22:10]:

Markdown has no way to communicate the character set in the document
(other than the Unicode Byte Order Marks, which is a generalized
property about text streams, not specific to Markdown)--and it would
be counterproductive to invent one. So that is a perfect example of
relevant metadata. And the second one, is how to turn it into
something else that the author wants. If it's not communicated, it's
going to be implied. Implied means "guessing" and likely "guessing
wrong".

Yet guessing wrong is largely without consequence.

There are really no syntax features that affect the document’s rendering
non-locally. If part of a document is written with unsupported syntax,
only that part will render incorrectly, but the other parts will come
out fine.

There are two use cases that I am particularly interested in:
#1 You put .md files in a project (readme.md, etc.). These .md files are 
then passed around among project users, which may include developers, 
copy-writers, copy-editors, etc. They need to be sure that the readme.md 
is treated in the same way, which ought to be communicated with the 
data. If one person edits the document in UTF-8 and commits and another 
person edits the document in ISO-8859-1, you're going to have problems.


#2 You have some app (let's say some web forum for example, but it 
literally could be anything, an electronic health record, some national 
criminal records, whatever) and you export data from the app. Say to 
some structured data format like XML or a sqlite database. Part of data 
liberation or backup or whatever. You want to get whatever your users 
actually input into the fields--not the HTMLized versions. So you need 
to annotate the blobs of data as Markdown, since users like to upload 
various kinds of data (Word docs, JPEG images, MP4 videos, bits of text 
like names of individuals, whatever).


In both cases, rendering matters "non-locally".



And there are no large overlapping surfaces among the syntaxes of the
various extensions (esp. those for very different document features),
which makes unsupported syntax unlikely to appear to have been intended
to be rendered as some completely dissimilar feature.


As someone new to Markdown development, I really want to see some 
comprehensive references (since "authority" in Markdown-land is notably 
absent). Besides, since Markdown is such a free-for-all, someone could 
easily write a Markdown processor that turns (!) into 
alert('hello!');.




So you will get a document that differs from the author’s intent in some
way. But it will be clear *where* the differences are and you will still
get all of the data in *some* form, quite possibly fully intelligible if
not pretty.
For what we might call "sensible flavors" of Markdown, yes. But the 
author's intent may be poorly represented when processed through a tool 
that injects lolcat pictures every third word. Or, the author's intent 
may be very well-represented.


The point is...we don't know what the author's intent is, /unless the 
author tells us/. And I think we need some more metadata to make the 
author's intent clear.




And because of the primary goal of Markdown to be human-readable in its
source form, there is always an easy and cheap last resort: view source.


This is a goal. Agreed.


Therefore the flavour parameter ought to be considered nothing more than
loosely informative, and the processor should just render the document
to the best of its ability regardless of the flavour specified. It MAY
use the parameter value to adapt to the document, in RFC 2119 lingo, but
ought not be bound by it.

I would reword this:

The flavor parameter informs recipients of the author's intent. The processor 
should just render the document to the best of its ability regardless of the 
flavor specified. It SHOULD
use the parameter value to adapt to the document.



I don't know what should happen if the flavor is absent. I am trying to 
understand. Let me put it this way: if you come across un-annotated 
Markdown in the wild (as in, not attached to any processing scripts, 
instructions, directions, whatever), what do you do? "Guess?"



Furthermore, an absent flavour parameter ought to mean that the flavour
is unspecified, not that it is any particular default flavour; i.e. the
choice of flavour in that case ought to be up to the processor.


The choice of how to act on the Markdown is /always/ up to the 
processor...so...probably. It just may not represent the author's intent.


Between this and the Gruber discussion, I need to get used to this idea 
that "guessing" is a normative part of Markdown culture. :)




Lastly, the spec should mention (as informal guidance to implementors)
that applications containing Markdown processors which have any chance
of being exposed to source documents of unknown flavour should, if at
all possible, provide a means for the user to view the source Markdown
d

Re: text/markdown effort in IETF (invite)

[Karl Dubost]

Aristotle,

Le 10 juil. 2014 à 12:06, Aristotle Pagaltzis  a écrit :
> Lastly, the spec should mention (as informal guidance to implementors)
> that applications containing Markdown processors which have any chance
> of being exposed to source documents of unknown flavour should, if at
> all possible, provide a means for the user to view the source Markdown
> document in unformatted form.

Very good point. Added to
https://github.com/karlcow/markdown-testsuite/issues/59


-- 
Karl Dubost 🐄
http://www.la-grange.net/karl/

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Aristotle Pagaltzis]

* Sean Leonard  [2014-07-09 22:10]:
> Markdown has no way to communicate the character set in the document
> (other than the Unicode Byte Order Marks, which is a generalized
> property about text streams, not specific to Markdown)--and it would
> be counterproductive to invent one. So that is a perfect example of
> relevant metadata. And the second one, is how to turn it into
> something else that the author wants. If it's not communicated, it's
> going to be implied. Implied means "guessing" and likely "guessing
> wrong".

Yet guessing wrong is largely without consequence.

There are really no syntax features that affect the document’s rendering
non-locally. If part of a document is written with unsupported syntax,
only that part will render incorrectly, but the other parts will come
out fine.

And there are no large overlapping surfaces among the syntaxes of the
various extensions (esp. those for very different document features),
which makes unsupported syntax unlikely to appear to have been intended
to be rendered as some completely dissimilar feature.

So you will get a document that differs from the author’s intent in some
way. But it will be clear *where* the differences are and you will still
get all of the data in *some* form, quite possibly fully intelligible if
not pretty.

And because of the primary goal of Markdown to be human-readable in its
source form, there is always an easy and cheap last resort: view source.

Bottom line, misrendering a document due to wrong choice of flavour is
annoying but inconsequential, due to the very nature of Markdown.

Therefore the flavour parameter ought to be considered nothing more than
loosely informative, and the processor should just render the document
to the best of its ability regardless of the flavour specified. It MAY
use the parameter value to adapt to the document, in RFC 2119 lingo, but
ought not be bound by it.

Furthermore, an absent flavour parameter ought to mean that the flavour
is unspecified, not that it is any particular default flavour; i.e. the
choice of flavour in that case ought to be up to the processor.

Lastly, the spec should mention (as informal guidance to implementors)
that applications containing Markdown processors which have any chance
of being exposed to source documents of unknown flavour should, if at
all possible, provide a means for the user to view the source Markdown
document in unformatted form.

Regards,
-- 
Aristotle Pagaltzis // 
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Fletcher T. Penney]

You appear to be testing MultiMarkdown in standard mode, rather than 
compatibility mode (`-c`).  If you're going to test against standard Markdown 
syntax you should use compatibility mode as it disables the additional features 
that alter the output (e.g. smart typography, anchors on headers, etc.).

When doing this properly, it appears that MMD fails only one test, which I will 
work on.

FTP

-- 
Fletcher T. Penney
fletc...@fletcherpenney.net 

On Jul 9, 2014, at 9:18 PM, Karl Dubost  wrote:

> Hi,
> 
> Michel Fortin said:
>> Markdown is in the spot where HTML was before HTML5 with each implementation 
>> doing its own thing. I don't know if Markdown will get out of there anytime 
>> soon.
> 
> Yes basically. And it's why Ciro Santilli has done an amazing work upon the 
> [test suite][1] I have started. 
> 
> [1]: https://github.com/karlcow/markdown-testsuite
> 
> The "issue" with Markdown and its flavors is that it is mainly used: 
> 
> * asan input format for something else, aka in a converter scenario
> * more than an exchange format with multiple emitters/consumers needing 
> interoperability.
> 
> It starts to change with the rise of some clients and this ties to the 
> mime-type "issue":
> 
>> I'll point out however that HTML never got anything like a "flavor" 
>> parameter in its MIME type, and even if it did it'd not have helped clear 
>> the mess in any way.
> 
> Yup agreed. A [MIME type][2] is useful in the case of "exchange format" when 
> an emitter and a receiver needs to understand what they are exchanging. In 
> the case of the input format, there is no issue because the environment is 
> constrained. When you play with multiple clients, the interoperability story 
> becomes interesting.
> 
> [2]: http://tools.ietf.org/html/draft-seantek-text-markdown-media-type-00
> 
> -- 
> Karl Dubost 🐄
> http://www.la-grange.net/karl/
> 
> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss



smime.p7s
Description: S/MIME cryptographic signature
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Karl Dubost]

Hi,

Michel Fortin said:
> Markdown is in the spot where HTML was before HTML5 with each implementation 
> doing its own thing. I don't know if Markdown will get out of there anytime 
> soon.

Yes basically. And it's why Ciro Santilli has done an amazing work upon the 
[test suite][1] I have started. 

[1]: https://github.com/karlcow/markdown-testsuite

The "issue" with Markdown and its flavors is that it is mainly used: 

* asan input format for something else, aka in a converter scenario
* more than an exchange format with multiple emitters/consumers needing 
interoperability.

It starts to change with the rise of some clients and this ties to the 
mime-type "issue":

> I'll point out however that HTML never got anything like a "flavor" parameter 
> in its MIME type, and even if it did it'd not have helped clear the mess in 
> any way.

Yup agreed. A [MIME type][2] is useful in the case of "exchange format" when an 
emitter and a receiver needs to understand what they are exchanging. In the 
case of the input format, there is no issue because the environment is 
constrained. When you play with multiple clients, the interoperability story 
becomes interesting.

[2]: http://tools.ietf.org/html/draft-seantek-text-markdown-media-type-00

-- 
Karl Dubost 🐄
http://www.la-grange.net/karl/

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Jason Davies]

While I don't disagree with these points, I don't think they are 
necessarily *the* point.


Markdown is -- sometime-- used as Markdown, by which I mean I read it 
raw and send it to people raw. But the vast majority of the time, it's a 
lightweight mark-up language and - most importantly - a transitional 
mode. It becomes something else (html, TeX, opml etc).


Its virtues are simplicity and adaptability. So, for instance, in 
Mailmate I can write in markdown and it will be interpreted into html 
rich text. But for that very reason, it doesn't implement numbered lists 
(the developer explained to me that this became problematic when you 
convert *back*, in a reply, which Mailmate has a good stab at.)


In other words, there are going to be reasons why someone might 
implement in differently for valid reasons (which you may or may not 
agree with). So you could say it's not true Markdown (well, it's not if 
you use Gruber's syntax). But its simplicity and growing popularity 
means that it's too tempting to use it: otherwise he would have to 
invent a parallel beta-code with a different syntax which is as 
frustrating as the way that different wikis use different mark-up  
(drives me nuts...I can never remember the different dialects).


So if you created a 'standard markdown', as HTML 5 did, you would also 
have a bunch of people who wouldn't implement it fully. HTML 5 was made 
mission critical by two things, it seems to me: 1) Microsoft's 
deliberate attempts to break HTML's universal rendering forced the 
community to unite and sort it out 2) the fact that massive commercial 
and social implications arose from websites not working properly. If IE 
had not been such a pain to code for, and the consequences of the minor 
variations were not great, you'd never have had HTML 5 -- there would 
not have been the will.


Markdown does not currently have that scenario. Its greatest asset is 
its relatively low-level specification and elegance. So, for instance, 
if you want to convert it to LaTeX or OPML, you can through 
multimarkdown. If your website, written in markdown, doesn't work 
properly, you just fix it because you have a standard to fix it against 
(thanks to 5).


So, without a strong impetus to enforce co-operation, If you created a 
new standard, you would *still* end up with one 'true' markdown and 
several variants which people would implement to suit their purposes -- 
which is precisely what you have now. There is a near-perfect 
specification, and there are variants.




There is not the urgency in this case: enforcement is therefore going to 
be a voluntary adherence to a single spec (or not). In other words, we 
are already wherever we are going to end up, with a few details changed.



. On 9 Jul 2014, at 20:06, Sean Leonard wrote:


Some realities are apparent, at least to me:
1. Markdown is a real thing. It's not plain text and it's not 
HTML--it's something different. (Heck, this list could be Markdown!)
2. People are using Markdown for real things of economic and social 
value.
3. Markdown is different from other _lightweight markup languages_. 
I.e., it's not reStructuredText, BBCode, javadocs, or Creole (wiki 
markup). But unlike the aforementioned examples, there is no authority 
that guides its development. (reStructuredText is a Python thing, for 
example, so the Python people are in charge of it.)
4. Things that are called Markdown (MultiMarkdown, GitHub Flavored 
Markdown, etc.) share more in common with each other than those in 
#3--therefore these things are related.
5. People are storing and exchanging Markdown-as-Markdown between 
systems. Not Markdown-as-plain-text, and not Markdown-as-HTML. Thus, 
there is a need for standardized interchange.
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Michel Fortin]

Le 9-juil.-2014 à 16:08, Sean Leonard  a écrit :

> The operating question is: What metadata (companion data) is /necessary/ to 
> reflect the creator's intent with respect to the data?
> 
> With Markdown, I think the answer is: you need the character set, and you 
> need to know how to turn the text into HTML (or XHTML, PDF, RTF, MS 
> Word/Office Open XML, or whatever).

Indeed.


> Markdown has no way to communicate the character set in the document (other 
> than the Unicode Byte Order Marks, which is a generalized property about text 
> streams, not specific to Markdown)--and it would be counterproductive to 
> invent one. So that is a perfect example of relevant metadata.

Fun fact: PHP Markdown is mostly encoding agnostic. It understands UTF-8 
sequences but any byte that is not a valid UTF-8 sequence is treated as a 
character in itself. It's only relevant when converting tabs into spaces 
however, and only if you have non-ASCII characters before the tab.

So whatever the input encoding is becomes the output's encoding (this works for 
HTML). Naturally, it's good to know the input's encoding if you want to know 
the output's. So obviously it's a good idea to specify the text encoding even 
though the parser itself doesn't need it, so you know the resulting document's 
encoding.

That's not really relevant though.


> And the second one, is how to turn it into something else that the author 
> wants. If it's not communicated, it's going to be implied. Implied means 
> "guessing" and likely "guessing wrong".

Ideally you'd use the exact same version of the same parser the author used to 
interpret the document in the first place.

Or you could be loose and use another version of the same parser.
Or you could be loose and use another parser claiming to be of the same flavor.
Or you could be loose and use another parser claiming to be of a superset of 
the given flavor.
Or you could be loose and use another Markdown parser.

It's a spectrum. Each step down will increase the likeliness of something going 
wrong.


> Hopefully this makes sense. I want to be more educated about this.

This makes perfect sense, but I fear there's no good answer to your second 
question. Since you want to know more, here's some insight.

It's important to understand that there is no notion of invalid Markdown input. 
As an implementer every time you fix what looks like a parsing bug to you or 
add a feature you're also breaking some valid input that was producing 
something else before. The implementer will usually only choose to break valid 
input that was deemed very unlikely to ever have been used before, but there's 
no way to know for sure (and no reliable way to measure impact either). So if 
you really really want to be sure things are parsed in the intended way, you 
should use the closest version possible of the same parser as the creator of 
the document was using.

Also, subtle changes can make things technically incompatible. For instance, 
Markdown Extra is mostly a superset of the original Markdown feature-wise, 
except for one small incompatible change: underscore emphasis within a word is 
disallowed. This was a deliberate change to fix some problems users were having 
with words that contained underscore. So even though most people would consider 
Markdown Extra as a superset of Markdown, it technically isn't. Other 
implementers might do the same thing but consider it as a bug fix instead and 
tell their users implementation implements the original syntax.

Babelmark 2 will tell you that implementations are pretty much evenly split on 
this:
http://johnmacfarlane.net/babelmark2/?normalize=1&text=word_with_emphasis

You'll even see that Pandoc implements both behaviour depending on whether 
you're in strict mode or not.

Something stranger happens with the shortcut reference syntax:
http://johnmacfarlane.net/babelmark2/?normalize=1&text=%5Blink%3F%5D%0A%0A%5Blink%3F%5D%3A+http%3A%2F%2Flink.x%2F

It's pretty much universally supported. It comes from a Markdown.pl beta that 
was never formally released but which is widely in use. If you were to go to 
the Markdown website and use the download link, you won't get the beta and it 
won't work. And while the second one is feature-wise a superset of the first, 
technically it could in some rare situations break documents, turning square 
bracketed text into links where it shouldn't:

Someone on [street Ivanhoe Carol][sIC] told me this:

> This is bad [sic].

[sIC]: http://sic.sickdomain

I sure wish things would be simpler. But as things are now, I have a hard time 
identifying what "flavor" could mean. Should "Markdown.pl-1.0.1" be a flavor on 
its own?


-- 
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

RE: text/markdown effort in IETF (invite)

[Waylan Limberg]

I think this comment [1] by Gruber on the mailing list in the past can shed
some light on what the spec is (is it markdown.pl, the syntax rules on
daringfireball.net, some mashup of various implementations, or something
else?). According to Gruber, it is the syntax rules and that's it. If that
is not "good enough" to get a mime-type, then I don’t think there is
anything else we can do.

[1]:
http://six.pairlist.net/pipermail/markdown-discuss/2008-February/001001.html

-Original Message-
From: Markdown-Discuss [mailto:markdown-discuss-boun...@six.pairlist.net] On
Behalf Of Sean Leonard
Sent: Wednesday, July 09, 2014 4:00 PM
To: markdown-discuss@six.pairlist.net
Subject: Re: text/markdown effort in IETF (invite)

On 7/9/2014 12:06 PM, Michel Fortin wrote:
> Le 9-juil.-2014 à 11:49, Sean Leonard  a écrit :
>
>> Hi markdown-discuss Folks:
>>
>> I am working on a Markdown effort in the Internet Engineering Task Force,
to standardize on "text/markdown" as the Internet media type for all
variations of Markdown content. You can read my draft here:
<http://tools.ietf.org/html/draft-seantek-text-markdown-media-type-00>.
>>
>> The proposal is already getting traction. Is there anyone on this 
>> list that is interested in participating or helping this effort? In 
>> particular we need to better understand and document what versions of 
>> Markdown exist, so that either Markdown as a family of informal 
>> syntaxes will start to converge, or if not, that Markdown variations 
>> have an easy way to be distinguished from one another. (See the 
>> "flavor" parameter discussed in the draft.)
> The "flavor" parameter is a good idea in theory. I'm not sure it'll be
very useful in general though. Nobody is going to annotate their file with
the right flavor unless there's a tangible benefit, and I don't see what the
benefit could be. Software that could do something useful with
markdown-identified content will likely ignore the flavor part when parsing
because no one wants to see "incompatible flavor" errors, especially when
commonly used parts of the syntax are compatible anyway.
>
> Markdown is in the spot where HTML was before HTML5 with each
implementation doing its own thing. I don't know if Markdown will get out of
there anytime soon. I'll point out however that HTML never got anything like
a "flavor" parameter in its MIME type, and even if it did it'd not have
helped clear the mess in any way.

Ok so here is where I really want to focus and learn some stuff from the
Markdown community. I am a fairly heavy Markdown user, but not a Markdown
developer or maintainer [yet].

 From what I have gathered of Markdown history as an observer, getting this
into one standardized syntax is like herding cats. Someone complains about
how _ and * interact, or how to do tables such-and-such a way, and then they
go off and write a different tool that does it differently. Ultimately it's
as much about style (which is very
personal) as functionality. To make matters worse, Gruber isn't providing
leadership, so anyone can write "Son of Markdown" and before you know it
we'll have 50 different standards. <http://xkcd.com/927/>

If someone else manages to write a formal specification, I am happy to refer
to that normatively in the (proposed) IETF work. If the community manages to
coalesce around that spec, I am also happy to refer to that spec as the
exclusive normative reference. But...you may as well boil the ocean. So I'm
not going to attempt that. I just want to call the ocean for what it is, and
it's not text/plain. :)

Sean

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sean Leonard]

On 7/9/2014 12:06 PM, Michel Fortin wrote:

Le 9-juil.-2014 à 11:49, Sean Leonard  a écrit :
The "flavor" parameter is a good idea in theory. [...] Nobody is going to 
annotate their file with the right flavor unless there's a tangible benefit[...]

[...] HTML never got anything like a "flavor" parameter in its MIME type, and 
even if it did it'd not have helped clear the mess in any way.


About this "flavors" thing. I know there are several lists floating out 
there of different Markdown implementations and variants (or if you 
don't like them being called Markdown, you can call them Illegitimate 
Sons of Markdown™). Which list is the most complete? Can someone show me 
(or make for the community) a really comprehensive list, and agree to 
update it?


When I wrote the -00 draft, I tried to follow the Media Type 
Registration Procedures. One requirement is to list required and 
optional parameters. Parameters are defined in RFC 6838 as "companion 
data". See RFC 6838 and in particular, Sections 1, 4.2.1, and 4.3.


All text/ types have at least one parameter: the charset. That is 
because all text data has to be interpreted according to a code (i.e., 
character set) that converts the bits of data into useful information. 
Nowadays we take Unicode (specifically UTF-8) for granted, but it's just 
not the case in reality. You can't just open a text file and hope for 
the best--you have to have /metadata/, express or implied, that tells 
you how to handle the blob of bits. The very fact that it is textual 
data has to be inferred from other things, such as the filename 
extension (when the data is in a file). A filename is just another piece 
of metadata.


When dealing with HTML, the charset could determined at least six ways:
1. as express external metadata, when the Content-Type has a charset 
parameter in the HTTP header.
2. as implied external metadata, when the HTTP header is absent but the 
client infers it from "other things" (e.g., the server, the IP address, 
or by looking at the ccTLD).
3. as express internal "metadata", with  or 
; or in the case of XHTML, encoding="iso-2022-jp"?>.
4. as express internal *data*, that is, the first bytes are 0xFF 0xFE 
(likely UTF-16LE), 0xFE 0xFF (likely UTF-16BE), or 0xEF 0xBB 0xBF 
(likely UTF-8).
5. as implied internal *data*, that is, "take the first 256 bytes and 
try to see if it decodes to something approximating HTML soup using some 
common character sets; if it fits, you quit".
6. as express user preference, that is, "I'm Japanese in Japan on a 
Windows machine, therefore on my browser, just assume everything is 
Shift-JIS".



See...there are all these crazy options...because nobody standardized on 
the character set when HTTP/HTML was developed; people assumed it was 
US-ASCII and then shoehorned lots of zany ways to make it something else.


At least with Markdown, we can probably safely eliminate #3 since 
Markdown is not intended to generate the  part of (X)HTML.


The operating question is: What metadata (companion data) is /necessary/ 
to reflect the creator's intent with respect to the data?


With Markdown, I think the answer is: you need the character set, and 
you need to know how to turn the text into HTML (or XHTML, PDF, RTF, MS 
Word/Office Open XML, or whatever).


Markdown has no way to communicate the character set in the document 
(other than the Unicode Byte Order Marks, which is a generalized 
property about text streams, not specific to Markdown)--and it would be 
counterproductive to invent one. So that is a perfect example of 
relevant metadata. And the second one, is how to turn it into something 
else that the author wants. If it's not communicated, it's going to be 
implied. Implied means "guessing" and likely "guessing wrong".


Hopefully this makes sense. I want to be more educated about this. Thanks!

Sean

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sean Leonard]

On 7/9/2014 12:06 PM, Michel Fortin wrote:

Le 9-juil.-2014 à 11:49, Sean Leonard  a écrit :


Hi markdown-discuss Folks:

I am working on a Markdown effort in the Internet Engineering Task Force, to standardize on 
"text/markdown" as the Internet media type for all variations of Markdown content. 
You can read my draft here: 
.

The proposal is already getting traction. Is there anyone on this list that is interested 
in participating or helping this effort? In particular we need to better understand and 
document what versions of Markdown exist, so that either Markdown as a family of informal 
syntaxes will start to converge, or if not, that Markdown variations have an easy way to 
be distinguished from one another. (See the "flavor" parameter discussed in the 
draft.)

The "flavor" parameter is a good idea in theory. I'm not sure it'll be very useful in 
general though. Nobody is going to annotate their file with the right flavor unless there's a 
tangible benefit, and I don't see what the benefit could be. Software that could do something 
useful with markdown-identified content will likely ignore the flavor part when parsing because no 
one wants to see "incompatible flavor" errors, especially when commonly used parts of the 
syntax are compatible anyway.

Markdown is in the spot where HTML was before HTML5 with each implementation doing its 
own thing. I don't know if Markdown will get out of there anytime soon. I'll point out 
however that HTML never got anything like a "flavor" parameter in its MIME 
type, and even if it did it'd not have helped clear the mess in any way.


Ok so here is where I really want to focus and learn some stuff from the 
Markdown community. I am a fairly heavy Markdown user, but not a 
Markdown developer or maintainer [yet].


From what I have gathered of Markdown history as an observer, getting 
this into one standardized syntax is like herding cats. Someone 
complains about how _ and * interact, or how to do tables such-and-such 
a way, and then they go off and write a different tool that does it 
differently. Ultimately it's as much about style (which is very 
personal) as functionality. To make matters worse, Gruber isn't 
providing leadership, so anyone can write "Son of Markdown" and before 
you know it we'll have 50 different standards. 


If someone else manages to write a formal specification, I am happy to 
refer to that normatively in the (proposed) IETF work. If the community 
manages to coalesce around that spec, I am also happy to refer to that 
spec as the exclusive normative reference. But...you may as well boil 
the ocean. So I'm not going to attempt that. I just want to call the 
ocean for what it is, and it's not text/plain. :)


Sean

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

RE: text/markdown effort in IETF (invite)

[Dennis E. Hamilton]

"Flavor" was handled in HTML with the DTD, FWIW.

-Original Message-
From: Markdown-Discuss [mailto:markdown-discuss-boun...@six.pairlist.net] On 
Behalf Of Michel Fortin
Sent: Wednesday, July 9, 2014 12:06
To: Discussion related to Markdown.
Subject: Re: text/markdown effort in IETF (invite)

[ ... ]
Markdown is in the spot where HTML was before HTML5 with each implementation 
doing its own thing. I don't know if Markdown will get out of there anytime 
soon. I'll point out however that HTML never got anything like a "flavor" 
parameter in its MIME type, and even if it did it'd not have helped clear the 
mess in any way.

-- 
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Fletcher T. Penney]

I disagree with the section quoted below.

To my knowledge, Gruber has not officially trademarked "Markdown".

Markdown was a word before Gruber used it, but for different contexts.

I am not a lawyer.

However, in the world of honest people, the word "Markdown" as applied 
to lightweight text formats belongs to Gruber.  Others may play off of 
it (PHP Markdown Extra, my own MultiMarkdown, etc.), but I can't create 
an entirely new syntax and call it Markdown.


FTP


On 7/9/14, 3:06 PM, Sean Leonard wrote:


I am of the same view. Anyone can call anything "Markdown"--no one is
stopping them. Just as anyone can call anything "ASCII art" or "mashups"
(i.e., there might be an ASCII standard but what people do with it is
totally different--it has become a cultural phenomenon). In the draft, I
restricted the eligible formats to "things based on John Gruber's
original Markdown tool and syntax from 2004".


--
Fletcher T. Penney
fletc...@fletcherpenney.net
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Michel Fortin]

Le 9-juil.-2014 à 11:49, Sean Leonard  a écrit :

> Hi markdown-discuss Folks:
> 
> I am working on a Markdown effort in the Internet Engineering Task Force, to 
> standardize on "text/markdown" as the Internet media type for all variations 
> of Markdown content. You can read my draft here: 
> .
> 
> The proposal is already getting traction. Is there anyone on this list that 
> is interested in participating or helping this effort? In particular we need 
> to better understand and document what versions of Markdown exist, so that 
> either Markdown as a family of informal syntaxes will start to converge, or 
> if not, that Markdown variations have an easy way to be distinguished from 
> one another. (See the "flavor" parameter discussed in the draft.)

The "flavor" parameter is a good idea in theory. I'm not sure it'll be very 
useful in general though. Nobody is going to annotate their file with the right 
flavor unless there's a tangible benefit, and I don't see what the benefit 
could be. Software that could do something useful with markdown-identified 
content will likely ignore the flavor part when parsing because no one wants to 
see "incompatible flavor" errors, especially when commonly used parts of the 
syntax are compatible anyway.

Markdown is in the spot where HTML was before HTML5 with each implementation 
doing its own thing. I don't know if Markdown will get out of there anytime 
soon. I'll point out however that HTML never got anything like a "flavor" 
parameter in its MIME type, and even if it did it'd not have helped clear the 
mess in any way.

-- 
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Sean Leonard]

Hello everyone,

On 7/9/2014 9:58 AM, Dennis E. Hamilton wrote:

I think the Internet draft is very clear.  It is not a Standards Track project. 
 It is a MIME-type registration proposal and the procedure for determination of 
flavors should satisfy whatever concerns there are.


That is correct. My purpose in creating this first draft is not to make 
a Markdown standard; it is to identify Markdown content in a 
standardized way, namely, with text/markdown.



In general, a MIME-type registration has to point to some place where there is 
a description of the format.  These are not particularly definitive or 
authoritative in some cases, and this registration could fail for lack of 
something definitive.  That is best dealt with on the IETF discussion list.


Yes, this is a sticking point. Experienced IETFers will raise (and have 
raised) concerns about the authoritative-ness of the format. But IETFers 
have less experience with Markdown compared to you all, which is why I'm 
bringing it up here (and elsewhere).




I have nothing to offer concerning "official" Markdown.  It would appear that 
the term has already been appropriated as a common noun and there is no means to protect 
against that being otherwise.


I am of the same view. Anyone can call anything "Markdown"--no one is 
stopping them. Just as anyone can call anything "ASCII art" or "mashups" 
(i.e., there might be an ASCII standard but what people do with it is 
totally different--it has become a cultural phenomenon). In the draft, I 
restricted the eligible formats to "things based on John Gruber's 
original Markdown tool and syntax from 2004".


Some realities are apparent, at least to me:
1. Markdown is a real thing. It's not plain text and it's not HTML--it's 
something different. (Heck, this list could be Markdown!)

2. People are using Markdown for real things of economic and social value.
3. Markdown is different from other _lightweight markup languages_. 
I.e., it's not reStructuredText, BBCode, javadocs, or Creole (wiki 
markup). But unlike the aforementioned examples, there is no authority 
that guides its development. (reStructuredText is a Python thing, for 
example, so the Python people are in charge of it.)
4. Things that are called Markdown (MultiMarkdown, GitHub Flavored 
Markdown, etc.) share more in common with each other than those in 
#3--therefore these things are related.
5. People are storing and exchanging Markdown-as-Markdown between 
systems. Not Markdown-as-plain-text, and not Markdown-as-HTML. Thus, 
there is a need for standardized interchange.


-Sean

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: text/markdown effort in IETF (invite)

[Jason Davies]

On 9 Jul 2014, at 17:07, Fletcher T. Penney wrote:

Unless it were to receive Gruber's blessing, it would have to be named 
something other than Markdown.


Really good summary Fletcher. I think unless someone steps up to create 
Son of Markdown as a project, we should all live with your third option 
(and I am just an inexpert user, would not be able to contribute any 
code/coding logic, so I'm not volunteering).


We could go classical for the name: the latin would be 'subscribe' which 
is not very helpful. (Very bad literal classical) Greek for Son of 
Markdown would be something like 'Hypographides', retaining the allusion 
to Mark-down without actually using Gruber's name.


The elusive acronym 'HGP' appeal to anyone?;)

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

RE: text/markdown effort in IETF (invite)

[Dennis E. Hamilton]

I think the Internet draft is very clear.  It is not a Standards Track project. 
 It is a MIME-type registration proposal and the procedure for determination of 
flavors should satisfy whatever concerns there are.  In general, a MIME-type 
registration has to point to some place where there is a description of the 
format.  These are not particularly definitive or authoritative in some cases, 
and this registration could fail for lack of something definitive.  That is 
best dealt with on the IETF discussion list.

I have nothing to offer concerning "official" Markdown.  It would appear that 
the term has already been appropriated as a common noun and there is no means 
to protect against that being otherwise.  


 -- Dennis E. Hamilton
dennis.hamil...@acm.org+1-206-779-9430
https://keybase.io/orcmid  PGP F96E 89FF D456 628A



-Original Message-
From: Markdown-Discuss [mailto:markdown-discuss-boun...@six.pairlist.net] On 
Behalf Of Fletcher T. Penney
Sent: Wednesday, July 9, 2014 09:08
To: Discussion related to Markdown.
Subject: Re: text/markdown effort in IETF (invite)

I would strongly recommend thinking through some of the "political" 
decisions before getting too far into this.

1) "Markdown" officially refers to the implementation and syntax created 
by John Gruber.

2) "Markdown" the perl implementation has not seen a bug fix in nearly 
10 years.

3) Gruber's voice has been noticeably absent from the list for a long 
time, except for a comment that I recall as basically saying that 
Markdown was essentially feature complete as far as he was concerned.

4) Gruber has specifically said in the past that new projects could not 
coopt the "Markdown" name and would have to be clearly disambiguated. 
For example, I would assume that anyone other than Gruber could not 
create "Markdown 2.0" to be the Markdown to rule them all...

5) I don't have numbers to back this up, but would strongly suspect that 
at this point very few people who think they use "Markdown" actually 
are.  Most are using various derivatives that have made wide-ranging 
decisions on how to handle edge cases, etc.  For most users, whose needs 
are very basic, the distinction is probably academic.  But I would 
suggest that these distinctions are very important when it comes to 
official standards.


I would propose that if there is to be an official standard based on 
"Markdown", it would first require defining what "Markdown" is.  To do 
that would (hopefully) require a more formalized description of the 
grammar.  If Gruber were to sign off on allowing this to use the 
"Markdown" name, fantastic.  But if not, a difficult decision would need 
to be made:

1) Build a standard based on Markdown.pl, bugs and all, and keep the 
"Markdown" name.

2) Develop a formalized version of the core syntax of Markdown, and base 
the standard on this.  Unless it were to receive Gruber's blessing, it 
would have to be named something other than Markdown.

3) Continue to use the term "Markdown" as a vague term that refers to a 
loosely related collection of tools, leaving users to wonder why a given 
document works with one tool, and not others.  At some point, a new 
common standard (e.g. "Son of Markdown" or whatever) may or may not 
arise that would require redefining all of this stuff.  Granted, efforts 
to organize such a standard have thus far failed despite multiple 
enthusiastic discussions over the years on this list.



My $.02


FTP



On 7/9/14, 11:49 AM, Sean Leonard wrote:
> Hi markdown-discuss Folks:
>
> I am working on a Markdown effort in the Internet Engineering Task
> Force, to standardize on "text/markdown" as the Internet media type for
> all variations of Markdown content. You can read my draft here:
> <http://tools.ietf.org/html/draft-seantek-text-markdown-media-type-00>.
>
> The proposal is already getting traction. Is there anyone on this list
> that is interested in participating or helping this effort? In
> particular we need to better understand and document what versions of
> Markdown exist, so that either Markdown as a family of informal syntaxes
> will start to converge, or if not, that Markdown variations have an easy
> way to be distinguished from one another. (See the "flavor" parameter
> discussed in the draft.)
>
> The draft is currently being discussed on apps-disc...@ietf.org.
>
> Kind regards,
>
> Sean Leonard
> Author of Markdown IETF Draft
> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss

-- 
Fletcher T. Penney
fletc...@fletcherpenney.net
___
Markdown-Discuss mailing list

Re: text/markdown effort in IETF (invite)

[Fletcher T. Penney]

I would strongly recommend thinking through some of the "political" 
decisions before getting too far into this.


1) "Markdown" officially refers to the implementation and syntax created 
by John Gruber.


2) "Markdown" the perl implementation has not seen a bug fix in nearly 
10 years.


3) Gruber's voice has been noticeably absent from the list for a long 
time, except for a comment that I recall as basically saying that 
Markdown was essentially feature complete as far as he was concerned.


4) Gruber has specifically said in the past that new projects could not 
coopt the "Markdown" name and would have to be clearly disambiguated. 
For example, I would assume that anyone other than Gruber could not 
create "Markdown 2.0" to be the Markdown to rule them all...


5) I don't have numbers to back this up, but would strongly suspect that 
at this point very few people who think they use "Markdown" actually 
are.  Most are using various derivatives that have made wide-ranging 
decisions on how to handle edge cases, etc.  For most users, whose needs 
are very basic, the distinction is probably academic.  But I would 
suggest that these distinctions are very important when it comes to 
official standards.



I would propose that if there is to be an official standard based on 
"Markdown", it would first require defining what "Markdown" is.  To do 
that would (hopefully) require a more formalized description of the 
grammar.  If Gruber were to sign off on allowing this to use the 
"Markdown" name, fantastic.  But if not, a difficult decision would need 
to be made:


1) Build a standard based on Markdown.pl, bugs and all, and keep the 
"Markdown" name.


2) Develop a formalized version of the core syntax of Markdown, and base 
the standard on this.  Unless it were to receive Gruber's blessing, it 
would have to be named something other than Markdown.


3) Continue to use the term "Markdown" as a vague term that refers to a 
loosely related collection of tools, leaving users to wonder why a given 
document works with one tool, and not others.  At some point, a new 
common standard (e.g. "Son of Markdown" or whatever) may or may not 
arise that would require redefining all of this stuff.  Granted, efforts 
to organize such a standard have thus far failed despite multiple 
enthusiastic discussions over the years on this list.




My $.02


FTP



On 7/9/14, 11:49 AM, Sean Leonard wrote:

Hi markdown-discuss Folks:

I am working on a Markdown effort in the Internet Engineering Task
Force, to standardize on "text/markdown" as the Internet media type for
all variations of Markdown content. You can read my draft here:
.

The proposal is already getting traction. Is there anyone on this list
that is interested in participating or helping this effort? In
particular we need to better understand and document what versions of
Markdown exist, so that either Markdown as a family of informal syntaxes
will start to converge, or if not, that Markdown variations have an easy
way to be distinguished from one another. (See the "flavor" parameter
discussed in the draft.)

The draft is currently being discussed on apps-disc...@ietf.org.

Kind regards,

Sean Leonard
Author of Markdown IETF Draft
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss


--
Fletcher T. Penney
fletc...@fletcherpenney.net
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

text/markdown effort in IETF (invite)

[Sean Leonard]

Hi markdown-discuss Folks:

I am working on a Markdown effort in the Internet Engineering Task 
Force, to standardize on "text/markdown" as the Internet media type for 
all variations of Markdown content. You can read my draft here: 
.


The proposal is already getting traction. Is there anyone on this list 
that is interested in participating or helping this effort? In 
particular we need to better understand and document what versions of 
Markdown exist, so that either Markdown as a family of informal syntaxes 
will start to converge, or if not, that Markdown variations have an easy 
way to be distinguished from one another. (See the "flavor" parameter 
discussed in the draft.)


The draft is currently being discussed on apps-disc...@ietf.org.

Kind regards,

Sean Leonard
Author of Markdown IETF Draft
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss