Re: Feature Request External label resolution

[Waylan Limberg]

On Sun, Apr 20, 2008 at 1:25 PM, Sherwood Botsford <[EMAIL PROTECTED]> wrote:
>
>  Umm.  Pointers to Markdown implementations with support for extensions?
>
>

[Python-Markdown][1] has an [extension api][2]. Interestingly my
[Abbreviation Extension][3] for Python-Markdown does something similar
 for abbreviations. Given an external file, it pulls all abbr defs and
uses them in processing the source file. It wouldn't be hard at all to
adapt it for link defs. What I really like about it is that the
external file can be any text document (markdown or otherwise) and the
limited parser will only extract the abbr defs (one per line) from it.

[1]: http://www.freewisdom.org/projects/python-markdown/
[2]: http://www.freewisdom.org/projects/python-markdown/Writing_Extensions
[3]: http://achinghead.com/markdown/abbr/


-- 

Waylan Limberg
[EMAIL PROTECTED]
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Michel Fortin]

Le 2008-04-20 à 11:47, Sherwood Botsford a écrit :


How do you deal with large numbers of links?


If you were using PHP Markdown, You could write a little extension  
like this:


class PreLinkedMarkdown_Parser extends Markdown_Parser {
var $preurls = array();
var $pretitles = array();
function PreLinkedMarkdown_Parser($preurls = array(),  
$pretitles = array()) {

parent::Markdown_Parser();
$this->$preurls = $preurls;
$this->pretitles = $pretitles;
$this->document_gamut["fillPreURLs"] = -999;
}
function fillPreURLs($text) {
$this->urls = $this->preurls;
$this->titles = $this->pretitles;
return $text;
}
}

and use your parser like this:

// read you link file once and populate arrays this way:
$preurls = array('link-ref' => 'url');
$pretitles = array('link-ref' => 'title'); // optional

$parser = new PreLinkedMarkdown_Parser($preurls, $prelinks);
$html = $parser->transform($text);

(Note that I haven't tested any of this code)

If there is enough interest, I could add this feature to the regular  
PHP Markdown Parser class.



Michel Fortin
[EMAIL PROTECTED]
http://michelf.com/


___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Milian Wolff]

Am Sonntag, 20. April 2008 schrieb Sherwood Botsford:
> Milian Wolff wrote:
> > Yes that seems to be a good idea. Though I recon you could pick one of
> > the Markdown implementations with support for extensions or one which is
> > object orientated so you could simply overload the basic link resolution.
> > Should be cleaner than hacking Markdown.pl
>
> Umm.  Pointers to Markdown implementations with support for
> extensions?

I'm only acquainted with PHP Markdown. It would be possible there.
-- 
Milian Wolff
http://milianw.de
OpenPGP key: CD1D1393


signature.asc
Description: This is a digitally signed message part.
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Sherwood Botsford]

Milian Wolff wrote:

Yes that seems to be a good idea. Though I recon you could pick one of the 
Markdown implementations with support for extensions or one which is object 
orientated so you could simply overload the basic link resolution. Should be 
cleaner than hacking Markdown.pl






Umm.  Pointers to Markdown implementations with support for 
extensions?



It's learning curve might be a bit steep though I think it is worth it. Maybe 
try it out for your next website.




I took a quick look at drupal.org.  Won't happen this spring.
While we still have snow on the ground, my busy season for
the tree farm starts in two weeks.  As soon as snow is gone I'll
be busier than a one handed piano player, with 4500 seedlings 
coming in.


So until next winter, I'll live with what I have.
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Milian Wolff]

Am Sonntag, 20. April 2008 schrieb Sherwood Botsford:
> Milian Wolff wrote:
> > Am Sonntag, 20. April 2008 schrieb Sherwood Botsford:
> >> So perhaps I should ask a more general question:
> >
> > You build your whole page with Markdown? No underlaying CMS or similar?
> > Especially if you consider that many links you really should think about
> > installing a CMS.
> >
> >> How do you deal with large numbers of links?
> >
> > I use Drupal and write the contents in Markdown. But Navigation, related
> > Links, Tags etc. are generated by Drupal - no Markdown there. The basic
> > templates don't use Markdown either.
>
> Currently I use template toolkit.  Markdown is used as a filter
> with tt2.
>
> The navigation is handled in two stages:
>
> 1.  A script called file_index runs find on my template
> directory.  It then reads in a file of sequence overrides.
> (The default is alphabetical order, which is fine for 90%
> of the files.  The overrides allow me to change this, or to
> mark regexes to be ignored.
>
> At the end of this, I have an index file of all pages that
> are to have menu items that is sorted in
> the order that menu items should appear.
>
> 2.  A template include does the actual menu build, whatever
> parts of the menu show for that page in the order they are
> in for that page. The menu at any given time shows the current
> page/folder and it's parents as being open, but all other
> folders are pruned.  Net result:  Even though there are a hundred
> plus pages in the site, the menu never shows more than about
> 15-20 items, and the present location is always indicated
> in a different color.  So menu is also my breadcrumb trail.
> (My pet peeve is sites where you chase links in circles.)
>
>
> Index files for each directory are of the form
>
> /Foo/Bar/Bar.html  That is, Directory Bar has a file Bar.html.
>
> This has some advantages for the site growing.
>
> E.g.  Initially I have Trees/Poplars.html
>
> When I get enough time to write the individual pages for the
> 5 kinds of poplars, this becomes
>
> Trees/Poplars/Poplars.html
> Trees/Poplars/Balsam_Poplar.html
> Trees/...
>
> Rerun file_index, rerun Ttree, and the links work.
>
> I really like markdown, because it doesn't get much in my
> way.  I want to keep markdown. Typically for me to create another
> page requires about 2 minutes plus the time to actually write the
> page.

This is a very interesting way of creating a website. Drupal and other CMS 
utilize PHP or other programming languages to keep things customizable and 
dynamic. Of course this requires a webserver with support for PHP or similar, 
most often a database like MySQL and so forth.

Since you already have a working system with your setup you should do what 
takes the least amount of work. And this would be:

> I may end up hacking markdown and disabling the link resolution
> mechanism, and create my own link resolution system as another
> ttree filter. Or hacking markdown to do a db lookup instead of
> a linear search through the file.

Yes that seems to be a good idea. Though I recon you could pick one of the 
Markdown implementations with support for extensions or one which is object 
orientated so you could simply overload the basic link resolution. Should be 
cleaner than hacking Markdown.pl

> So Markdown at present only sees the links that are in my content
> pages.  And so right now there are only a dozen or so.  But
> as my site grows, the importance of jumping from page to page
> via internal references will increase dramatically.
>
> I'll take a look at Drupal.

It's learning curve might be a bit steep though I think it is worth it. Maybe 
try it out for your next website.


-- 
Milian Wolff
http://milianw.de
OpenPGP key: CD1D1393


signature.asc
Description: This is a digitally signed message part.
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Aristotle Pagaltzis]

* Tomas Doran <[EMAIL PROTECTED]> [2008-04-20 10:55]:
> If you wanted to add an accessor for the URLs hashref on the
> module, and an option to the Markdown.pl script that I bundle
> in the distribution to give it this functionality, I'd be very
> happy to take a patch. :_)

That would have been great to have for me a while ago. I had to
ditch Markdown in favour of WYSIWYG anyway, so it’s no longer an
itch for me. But if we were still using it, I’d want such an
accessor: at this point we’d be appending several thousand link
references to every page, which is not exactly cheap.

Regards,
-- 
Aristotle Pagaltzis // 
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Sherwood Botsford]

Milian Wolff wrote:

Am Sonntag, 20. April 2008 schrieb Sherwood Botsford:

So perhaps I should ask a more general question:


You build your whole page with Markdown? No underlaying CMS or similar? 
Especially if you consider that many links you really should think about 
installing a CMS.



How do you deal with large numbers of links?


I use Drupal and write the contents in Markdown. But Navigation, related 
Links, Tags etc. are generated by Drupal - no Markdown there. The basic 
templates don't use Markdown either.





Currently I use template toolkit.  Markdown is used as a filter
with tt2.

The navigation is handled in two stages:

1.  A script called file_index runs find on my template 
directory.  It then reads in a file of sequence overrides.

(The default is alphabetical order, which is fine for 90%
of the files.  The overrides allow me to change this, or to
mark regexes to be ignored.

At the end of this, I have an index file of all pages that
are to have menu items that is sorted in
the order that menu items should appear.

2.  A template include does the actual menu build, whatever
parts of the menu show for that page in the order they are
in for that page. The menu at any given time shows the current
page/folder and it's parents as being open, but all other
folders are pruned.  Net result:  Even though there are a hundred 
plus pages in the site, the menu never shows more than about 
15-20 items, and the present location is always indicated

in a different color.  So menu is also my breadcrumb trail.
(My pet peeve is sites where you chase links in circles.)


Index files for each directory are of the form

/Foo/Bar/Bar.html  That is, Directory Bar has a file Bar.html.

This has some advantages for the site growing.

E.g.  Initially I have Trees/Poplars.html

When I get enough time to write the individual pages for the
5 kinds of poplars, this becomes

Trees/Poplars/Poplars.html
Trees/Poplars/Balsam_Poplar.html
Trees/...

Rerun file_index, rerun Ttree, and the links work.

I really like markdown, because it doesn't get much in my
way.  I want to keep markdown. Typically for me to create another
page requires about 2 minutes plus the time to actually write the
page.

I may end up hacking markdown and disabling the link resolution
mechanism, and create my own link resolution system as another
ttree filter. Or hacking markdown to do a db lookup instead of
a linear search through the file.

So Markdown at present only sees the links that are in my content 
pages.  And so right now there are only a dozen or so.  But

as my site grows, the importance of jumping from page to page
via internal references will increase dramatically.

I'll take a look at Drupal.
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Milian Wolff]

Am Sonntag, 20. April 2008 schrieb Sherwood Botsford:
> So perhaps I should ask a more general question:

You build your whole page with Markdown? No underlaying CMS or similar? 
Especially if you consider that many links you really should think about 
installing a CMS.

> How do you deal with large numbers of links?

I use Drupal and write the contents in Markdown. But Navigation, related 
Links, Tags etc. are generated by Drupal - no Markdown there. The basic 
templates don't use Markdown either.

-- 
Milian Wolff
http://milianw.de
OpenPGP key: CD1D1393


signature.asc
Description: This is a digitally signed message part.
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Sherwood Botsford]

Fletcher T. Penney wrote:

You could do something like:

cat myfile.txt refs.txt | MultiMarkdown.pl > myfile.html


F-

On Apr 20, 2008, at 10:40 AM, John MacFarlane wrote:

Pandoc concatenates input from all files specified on the command
line.  So you can just do:

   pandoc myfile.txt refs.txt > myfile.html

Seems to me that this would be a reasonable default behavior for
Markdown.pl as well, but it doesn't seem to work that way now.

John







___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss


I've had several people who have suggested this as a valid
solution.  It makes sense if you have a few dozen links.
And if I was sufficiently clever, I could break the site down
so that references were in a bunch of refs.txt files, and no
refs.txt file would have mroe than a few dozen links.

However I don't know how to partition my site in such a way.

If refs.txt has 2000 links in it, and every file
has to parse the entire refs file, it takes a long time.
As the site grows, processing time will grow quadratically.
(If I double the number of files, it will also double the number 
of links.  So a site that is twice as big will have

also have twice as many links.

At present I have a small site with 117 pages and 183 links.
Given that  In a year I figure the link count will be over a 
thousand.


At present I have one file that has a 1000 line table (including
all the tags on separate lines)  When TemplateToolkit / markdown
hit this file there is a 10 - 15 second pause.  Since TT is doing
it as an INSERT, not as an INCLUDE, TT isn't even looking at the 
file, so I think it's Markdown scanning this file looking for

tags that is causing the delay.  If a thousand line file with no
partial matches is slowing down Markdown this much, I would
expect that a file with 997 non-matching label/url lines and 3
matching label/url lines would cause considerably greater delay.

So perhaps I should ask a more general question:

How do you deal with large numbers of links?
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Fletcher T. Penney]

You could do something like:

cat myfile.txt refs.txt | MultiMarkdown.pl > myfile.html


F-

On Apr 20, 2008, at 10:40 AM, John MacFarlane wrote:

Pandoc concatenates input from all files specified on the command
line.  So you can just do:

   pandoc myfile.txt refs.txt > myfile.html

Seems to me that this would be a reasonable default behavior for
Markdown.pl as well, but it doesn't seem to work that way now.

John




--
Fletcher T. Penney
[EMAIL PROTECTED]

If God dropped acid, would he see people?
- Steven Wright



smime.p7s
Description: S/MIME cryptographic signature
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[John MacFarlane]

Pandoc concatenates input from all files specified on the command
line.  So you can just do:

pandoc myfile.txt refs.txt > myfile.html

Seems to me that this would be a reasonable default behavior for
Markdown.pl as well, but it doesn't seem to work that way now.

John

+++ Sherwood Botsford [Apr 19 08 16:28 ]:
> One of the things I'm coming up against.  Maintaining a non-small
> web site with many internal links is a pain.
>
> Consider:
>
> Suppose that at one point I have
>
> site/
>   Images
>   Business
>   Home
>   ...
>
> Later the site gets more complex, and Images has a bunch of sub  
> directories.
> site/
>   Images
> header_rotate
> inventory_pix
> misc
>   Business
>   Home
>
> When this happens I have to change the link for every pic
> on every page.  If I use an image in 6 places, I have to
> change it in 6 places.
>
> HOWEVER
>
> Suppose I cleverly used the footnote form of links.
>
> I.e:
>
> [Image alt text][LABEL]
>
> Suppose that markdown was clever enough to reference an external
> file (in .markdownrc of course) for the resolution of LABEL.
>
> NOW when I re-arrange the universe, I only have to change the reference 
> in this one file, NOT in every file that references it.
>
> ___
> Markdown-Discuss mailing list
> Markdown-Discuss@six.pairlist.net
> http://six.pairlist.net/mailman/listinfo/markdown-discuss
>
___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss

Re: Feature Request External label resolution

[Tomas Doran]

On 20 Apr 2008, at 07:11, Sherwood Botsford wrote:


Hmm.  Ok, I think I could to it this way in Template Toolkit:





Downside is then that Markdown has to process every reference
which if you have a few thousand will be time consuming.
(I have an INSERT text at one point that jsut in template toolkit
takes 10-12 seconds to do. It's only a thousand lines, but there
is zero processing to do, it just stuffs it into the file.


If you're prepared to have a fiddle with perl, have a look at  
Text::Markdown (on CPAN)..


You can do this reasonably trivially - here is a ghetto version of  
Markdown.pl which will suck all your link references out of  
~/.markdownrc:


use strict;
use warnings;
use Text::Markdown;
use File::Slurp;
use File::HomeDir;
use Path::Class;

my $m = Text::Markdown->new;
my $urls;
{
my $markdownrc = read_file(file(File::HomeDir->my_home,  
'.markdownrc'));

$m->markdown($markdownrc);
$urls = $m->{_urls};
}

print $m->markdown(<>, {urls => $urls});

That should do what you originally asked for - adjust to taste..

If you wanted to add an accessor for the URLs hashref on the module,  
and an option to the Markdown.pl script that I bundle in the  
distribution to give it this functionality, I'd be very happy to take  
a patch. :_)


Cheers
Tom

___
Markdown-Discuss mailing list
Markdown-Discuss@six.pairlist.net
http://six.pairlist.net/mailman/listinfo/markdown-discuss