Re: Markdown resource provider
On Sat, 2018-07-28 at 13:28 -0400, Jason E Bailey wrote: > I believe there's a difference between what the two end goals are. > There is a the rendering of a Markdown resource into HTML and then > there is using a Markdown file to generate a resource. That is a good distinction to be made, and one of the things which I've tried to reflect on the most. Bertrand's suggestion of distinct fields, lazily computed, for 'raw' vs 'rendered' would solve this nicely and I'm going to investigate this. > > A couple of thoughts on this. > # For a Markdown Resource providing attributes, I don't see a reason > to use YAML for this. Markdown support integrated HTML and HTML > supports custom tags. You could create some thing along the lines > of this wouldn't be > displayed when viewing a rendered version of the file but be > accessible when parsing. I think we clarified this in the email thread - YAML front matter is supported by other tools (and also nicely by Github for viewing) so IMO it's a good argument for adpoting it. Thanks, Robert
Re: Markdown resource provider
On Fri, 2018-07-27 at 15:49 +0200, Bertrand Delacretaz wrote: > On Fri, Jul 27, 2018 at 3:16 PM Daniel Klco > wrote: > > On Fri, Jul 27, 2018, 8:46 AM Jason E Bailey > > wrote: > > ... > > > Not understanding the phrase "front matter" in it's usage here > > > > I'm assuming YAML front matter like is used in Jekyll. Correct? .. > > That's my understanding as well, like the first few fields at the top > of > https://raw.githubusercontent.com/apache/sling-site/master/src/main/jbake/content/contributing.md Exactly. You can also see it in https://github.com/apache/sling-whiteboard/blob/master/mdresourceprovider/src/test/resources/md-test/index.md Robert
Re: Markdown resource provider
On Fri, 2018-07-27 at 10:14 +0200, Andrei Dulvac wrote: > Hi. > > That's cool indeed! > > What about using (potentially different) HTL scripts and create a > Sling > model that reads the raw markdown? The markdown files have a default resource type of sling/markdown/file [1] . This can be overriden by defining another 'sling:resourceType' property in the YAML front matter. And using Bertrand's suggestion of separating 'raw' and 'rendered' properties means that we can potentially create independend rendering scripts, and not rely on the one from the MarkdownResource. Thanks, Robert [1]: https://github.com/apache/sling-whiteboard/blob/f96ac30acce34426a46cffb85dacc89d7801255b/mdresourceprovider/src/main/java/org/apache/sling/mdresource/impl/MarkdownResource.java#L123
Re: Markdown resource provider
On Fri, 2018-07-27 at 08:07 +0200, Bertrand Delacretaz wrote: > On Fri, Jul 27, 2018 at 1:18 AM Robert Munteanu > wrote: > ... > > - render everything else as HTML into jcr:description ... > > I think rendering is a good idea but how about using a more specific > property name? > > Maybe use sling:markdownBody and sling:renderedHtmlBody with the > latter being lazily rendered? Or maybe more concise names, but > specific names would help avoid surprises IMO. Those are good points, will change from jcr:title and jcr:description and investigate having separate 'rendered' vs 'raw' properties. Thanks! Robert
Re: Markdown resource provider
On Fri, 2018-07-27 at 07:51 +0200, Carsten Ziegeler wrote: > Interesting, I've started similar work some time back but never > finished > it :) Back then I thought that this is a cross cutting concern, md > files > can be stored anywhere, in the file system, in jcr, fetched by a > special > github provider etc. Therefore I started with implementing a resource > decorator instead which was configured with a path prefix. All > resources > beneath that path with the extension md where "decorated" in a > similar > way as you describe. Hm, today I learned something now :-) I will look into the ResourceDecorator mechanism. > > What are you using for the md to html transformation? Flexmark-java [1] with the YAML front matter extension. [1]: https://github.com/vsch/flexmark-java
Re: Markdown resource provider
on a more serious note, this would be possibly great for dita as well (for example better process for the xmladdon for AEM) Ruben On 7/29/2018 6:30 AM, Jason E Bailey wrote: Eugen - I didn't realize that it was a thing. So I'm good with it. However I now have a strong desire to make an HTML resource provider that will define resources trees with HTML, which you can then use to generate HTML. A meta-HTML :) - Jason On Sun, Jul 29, 2018, at 1:52 AM, Ioan Eugen Stan wrote: Hi, Nice work Robert. @Jason: I do believe there is value in yaml frontmatter. A lot of existing tools support markdown + frontmatter and people using them are familiar with that. Adding something Sling specific is going to make interoerability harder. It is going to be non portable as well. I'm in favor of using front matter. It will help port markdown apps to sling and promote interoperability between such apps. Regards, Eugen Stan Netdava International Mesaj original De la: jason.bai...@24601.org Trimis: 28 iulie 2018 20:28 Către: dev@sling.apache.org Răsp. la: dev@sling.apache.org Subiect: Re: Markdown resource provider I believe there's a difference between what the two end goals are. There is a the rendering of a Markdown resource into HTML and then there is using a Markdown file to generate a resource. A couple of thoughts on this. # For a Markdown Resource providing attributes, I don't see a reason to use YAML for this. Markdown support integrated HTML and HTML supports custom tags. You could create some thing along the lines of this wouldn't be displayed when viewing a rendered version of the file but be accessible when parsing. # We have a module for taking different file types and creating a Resource Object out of them, that's the ContentParser. If the goal was to import the md file into an existing resource we'd use the ContentLoader. If we wanted the ability to read and write the file then I see a ResourceProvider and since we have a FSResourceProvider I still think it makes sense to allow that to be expandable. -- Jason On Fri, Jul 27, 2018, at 10:31 AM, Carsten Ziegeler wrote: Daniel Klco wrote My first thought after reading the last paragraph was - Wouldn't it be cool if the FsResourceProvider was extensible so that specific files could be rendered in a specific manner, then you could add a MarkDown Handler and it would make it easier for other people to add custom handlers or to extend existing one for specific requirements. Agreed! This mechanism could be useful for XML or JSON files as well. Imagine if one could register an XSLT handler and just have Sling serve the rendered HTML for a dump of XML files. That's why I mentioned the resource decorator approach - it allows you to do this, and then you're even independent of the resource provider serving the resource. The decorator approach might not be the most obvious one, but I think it does the trick and doesn't require us to add another overlapping concept. Regards Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Markdown resource provider
Eugen - I didn't realize that it was a thing. So I'm good with it. However I now have a strong desire to make an HTML resource provider that will define resources trees with HTML, which you can then use to generate HTML. A meta-HTML :) - Jason On Sun, Jul 29, 2018, at 1:52 AM, Ioan Eugen Stan wrote: > Hi, > > Nice work Robert. > > @Jason: I do believe there is value in yaml frontmatter. A lot of > existing tools support markdown + frontmatter and people using them are > familiar with that. Adding something Sling specific is going to make > interoerability harder. It is going to be non portable as well. > > I'm in favor of using front matter. It will help port markdown apps to > sling and promote interoperability between such apps. > > Regards, > > Eugen Stan > Netdava International > > Mesaj original > De la: jason.bai...@24601.org > Trimis: 28 iulie 2018 20:28 > Către: dev@sling.apache.org > Răsp. la: dev@sling.apache.org > Subiect: Re: Markdown resource provider > > I believe there's a difference between what the two end goals are. There > is a the rendering of a Markdown resource into HTML and then there is > using a Markdown file to generate a resource. > > A couple of thoughts on this. > # For a Markdown Resource providing attributes, I don't see a reason to > use YAML for this. Markdown support integrated HTML and HTML supports > custom tags. You could create some thing along the lines of props data-sling-resourceType="foo"> this wouldn't be displayed when > viewing a rendered version of the file but be accessible when parsing. > > # We have a module for taking different file types and creating a > Resource Object out of them, that's the ContentParser. If the goal was > to import the md file into an existing resource we'd use the > ContentLoader. If we wanted the ability to read and write the file then > I see a ResourceProvider and since we have a FSResourceProvider I still > think it makes sense to allow that to be expandable. > > > > -- > Jason > > On Fri, Jul 27, 2018, at 10:31 AM, Carsten Ziegeler wrote: > > > > > > > > > Daniel Klco wrote > > >> My first thought after reading the last paragraph was - Wouldn't it be > > >> cool if the FsResourceProvider was extensible so that specific files > > >> could > > >> be rendered in a specific manner, then you could add a MarkDown Handler > > >> and > > >> it would make it easier for other people to add custom handlers or to > > >> extend existing one for specific requirements. > > >> > > > > > > Agreed! This mechanism could be useful for XML or JSON files as well. > > > Imagine if one could register an XSLT handler and just have Sling serve > > > the > > > rendered HTML for a dump of XML files. > > > > > > > That's why I mentioned the resource decorator approach - it allows you > > to do this, and then you're even independent of the resource provider > > serving the resource. > > The decorator approach might not be the most obvious one, but I think it > > does the trick and doesn't require us to add another overlapping concept. > > > > Regards > > Carsten > > -- > > Carsten Ziegeler > > Adobe Research Switzerland > > cziege...@apache.org
Re: Markdown resource provider
Hi, Nice work Robert. @Jason: I do believe there is value in yaml frontmatter. A lot of existing tools support markdown + frontmatter and people using them are familiar with that. Adding something Sling specific is going to make interoerability harder. It is going to be non portable as well. I'm in favor of using front matter. It will help port markdown apps to sling and promote interoperability between such apps. Regards, Eugen Stan Netdava International Mesaj original De la: jason.bai...@24601.org Trimis: 28 iulie 2018 20:28 Către: dev@sling.apache.org Răsp. la: dev@sling.apache.org Subiect: Re: Markdown resource provider I believe there's a difference between what the two end goals are. There is a the rendering of a Markdown resource into HTML and then there is using a Markdown file to generate a resource. A couple of thoughts on this. # For a Markdown Resource providing attributes, I don't see a reason to use YAML for this. Markdown support integrated HTML and HTML supports custom tags. You could create some thing along the lines of this wouldn't be displayed when viewing a rendered version of the file but be accessible when parsing. # We have a module for taking different file types and creating a Resource Object out of them, that's the ContentParser. If the goal was to import the md file into an existing resource we'd use the ContentLoader. If we wanted the ability to read and write the file then I see a ResourceProvider and since we have a FSResourceProvider I still think it makes sense to allow that to be expandable. -- Jason On Fri, Jul 27, 2018, at 10:31 AM, Carsten Ziegeler wrote: > > > > > Daniel Klco wrote > >> My first thought after reading the last paragraph was - Wouldn't it be > >> cool if the FsResourceProvider was extensible so that specific files could > >> be rendered in a specific manner, then you could add a MarkDown Handler and > >> it would make it easier for other people to add custom handlers or to > >> extend existing one for specific requirements. > >> > > > > Agreed! This mechanism could be useful for XML or JSON files as well. > > Imagine if one could register an XSLT handler and just have Sling serve the > > rendered HTML for a dump of XML files. > > > > That's why I mentioned the resource decorator approach - it allows you > to do this, and then you're even independent of the resource provider > serving the resource. > The decorator approach might not be the most obvious one, but I think it > does the trick and doesn't require us to add another overlapping concept. > > Regards > Carsten > -- > Carsten Ziegeler > Adobe Research Switzerland > cziege...@apache.org
Re: Markdown resource provider
I believe there's a difference between what the two end goals are. There is a the rendering of a Markdown resource into HTML and then there is using a Markdown file to generate a resource. A couple of thoughts on this. # For a Markdown Resource providing attributes, I don't see a reason to use YAML for this. Markdown support integrated HTML and HTML supports custom tags. You could create some thing along the lines of this wouldn't be displayed when viewing a rendered version of the file but be accessible when parsing. # We have a module for taking different file types and creating a Resource Object out of them, that's the ContentParser. If the goal was to import the md file into an existing resource we'd use the ContentLoader. If we wanted the ability to read and write the file then I see a ResourceProvider and since we have a FSResourceProvider I still think it makes sense to allow that to be expandable. -- Jason On Fri, Jul 27, 2018, at 10:31 AM, Carsten Ziegeler wrote: > > > > > Daniel Klco wrote > >> My first thought after reading the last paragraph was - Wouldn't it be > >> cool if the FsResourceProvider was extensible so that specific files could > >> be rendered in a specific manner, then you could add a MarkDown Handler and > >> it would make it easier for other people to add custom handlers or to > >> extend existing one for specific requirements. > >> > > > > Agreed! This mechanism could be useful for XML or JSON files as well. > > Imagine if one could register an XSLT handler and just have Sling serve the > > rendered HTML for a dump of XML files. > > > > That's why I mentioned the resource decorator approach - it allows you > to do this, and then you're even independent of the resource provider > serving the resource. > The decorator approach might not be the most obvious one, but I think it > does the trick and doesn't require us to add another overlapping concept. > > Regards > Carsten > -- > Carsten Ziegeler > Adobe Research Switzerland > cziege...@apache.org
Re: Markdown resource provider
> Daniel Klco wrote >> My first thought after reading the last paragraph was - Wouldn't it be >> cool if the FsResourceProvider was extensible so that specific files could >> be rendered in a specific manner, then you could add a MarkDown Handler and >> it would make it easier for other people to add custom handlers or to >> extend existing one for specific requirements. >> > > Agreed! This mechanism could be useful for XML or JSON files as well. > Imagine if one could register an XSLT handler and just have Sling serve the > rendered HTML for a dump of XML files. > That's why I mentioned the resource decorator approach - it allows you to do this, and then you're even independent of the resource provider serving the resource. The decorator approach might not be the most obvious one, but I think it does the trick and doesn't require us to add another overlapping concept. Regards Carsten -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Re: Markdown resource provider
On Fri, Jul 27, 2018 at 3:16 PM Daniel Klco wrote: > On Fri, Jul 27, 2018, 8:46 AM Jason E Bailey wrote: ... > > Not understanding the phrase "front matter" in it's usage here > I'm assuming YAML front matter like is used in Jekyll. Correct? .. That's my understanding as well, like the first few fields at the top of https://raw.githubusercontent.com/apache/sling-site/master/src/main/jbake/content/contributing.md -Bertrand
Re: Markdown resource provider
On Fri, Jul 27, 2018, 8:46 AM Jason E Bailey wrote: > In line > > - Jason > > On Thu, Jul 26, 2018, at 7:18 PM, Robert Munteanu wrote: > > Hi, > > > > I've been toying around with a Markdown resource provider in the > > whiteboard [0]. It's very much a work-in-progress but since it does > > something - aka render Markdown files, including support for arbitrary > > Sling properties, I would like to discuss a couple of design decisions. > > > > 1. The resource is created from a markdown file in the following way: > > > > - if a first level heading is found, place the text into jcr:title > > - if a YAML front matter is found, save the properties in the resource > > properties > > Not understanding the phrase "front matter" in it's usage here > I'm assuming YAML front matter like is used in Jekyll. Correct? > > - render everything else as HTML into jcr:description > > > > I am not sure that rendering the markdown in the resource provider is > > the best way. On one hand, it keeps everything simple and does not > > require special work in the rendering script. On the other hand, > > rendering JSON/TXT with HTML inside is not that nice. > > I think it depends on the use case. From my experience there is already a > lot of text strings with HTML inside of it, so I don't see this as > something unique. Not looking at the code yet, but I assume that this is > being rendered when the resource is being requested. Which is something > that I would prefer. > > > > > 2. Only files with the 'md' extension are handled, all others are > > ignored. > > > > To generate static sites, usually markdown files are mixed with images, > > css, js files. Does it make sense to 'pass-through' these files and > > expose them as plain files? There could be some ties with the > > FsResourceProvider. Alternatively the static assets can be placed in a > > different subtree. > > > > Thoughts? > > My first thought after reading the last paragraph was - Wouldn't it be > cool if the FsResourceProvider was extensible so that specific files could > be rendered in a specific manner, then you could add a MarkDown Handler and > it would make it easier for other people to add custom handlers or to > extend existing one for specific requirements. > Agreed! This mechanism could be useful for XML or JSON files as well. Imagine if one could register an XSLT handler and just have Sling serve the rendered HTML for a dump of XML files. > > > Thanks, > > > > Robert > > > > [0]: > > > https://github.com/apache/sling-whiteboard/tree/master/mdresourceprovider >
Re: Markdown resource provider
In line - Jason On Thu, Jul 26, 2018, at 7:18 PM, Robert Munteanu wrote: > Hi, > > I've been toying around with a Markdown resource provider in the > whiteboard [0]. It's very much a work-in-progress but since it does > something - aka render Markdown files, including support for arbitrary > Sling properties, I would like to discuss a couple of design decisions. > > 1. The resource is created from a markdown file in the following way: > > - if a first level heading is found, place the text into jcr:title > - if a YAML front matter is found, save the properties in the resource > properties Not understanding the phrase "front matter" in it's usage here > - render everything else as HTML into jcr:description > > I am not sure that rendering the markdown in the resource provider is > the best way. On one hand, it keeps everything simple and does not > require special work in the rendering script. On the other hand, > rendering JSON/TXT with HTML inside is not that nice. I think it depends on the use case. From my experience there is already a lot of text strings with HTML inside of it, so I don't see this as something unique. Not looking at the code yet, but I assume that this is being rendered when the resource is being requested. Which is something that I would prefer. > > 2. Only files with the 'md' extension are handled, all others are > ignored. > > To generate static sites, usually markdown files are mixed with images, > css, js files. Does it make sense to 'pass-through' these files and > expose them as plain files? There could be some ties with the > FsResourceProvider. Alternatively the static assets can be placed in a > different subtree. > > Thoughts? My first thought after reading the last paragraph was - Wouldn't it be cool if the FsResourceProvider was extensible so that specific files could be rendered in a specific manner, then you could add a MarkDown Handler and it would make it easier for other people to add custom handlers or to extend existing one for specific requirements. > Thanks, > > Robert > > [0]: > https://github.com/apache/sling-whiteboard/tree/master/mdresourceprovider
Re: Markdown resource provider
Hi. That's cool indeed! What about using (potentially different) HTL scripts and create a Sling model that reads the raw markdown? - Andrei On Fri, Jul 27, 2018, 08:07 Bertrand Delacretaz wrote: > Hi Robert, > > Cool stuff! > > On Fri, Jul 27, 2018 at 1:18 AM Robert Munteanu > wrote: > ... > > - render everything else as HTML into jcr:description ... > > I think rendering is a good idea but how about using a more specific > property name? > > Maybe use sling:markdownBody and sling:renderedHtmlBody with the > latter being lazily rendered? Or maybe more concise names, but > specific names would help avoid surprises IMO. > > > ...To generate static sites, usually markdown files are mixed with > images, > > css, js files. Does it make sense to 'pass-through' these files and > > expose them as plain files?... > > I think so, and maybe Carsten's decorator approach helps achieve that. > > -Bertrand >
Re: Markdown resource provider
Hi Robert, Cool stuff! On Fri, Jul 27, 2018 at 1:18 AM Robert Munteanu wrote: ... > - render everything else as HTML into jcr:description ... I think rendering is a good idea but how about using a more specific property name? Maybe use sling:markdownBody and sling:renderedHtmlBody with the latter being lazily rendered? Or maybe more concise names, but specific names would help avoid surprises IMO. > ...To generate static sites, usually markdown files are mixed with images, > css, js files. Does it make sense to 'pass-through' these files and > expose them as plain files?... I think so, and maybe Carsten's decorator approach helps achieve that. -Bertrand
Re: Markdown resource provider
Interesting, I've started similar work some time back but never finished it :) Back then I thought that this is a cross cutting concern, md files can be stored anywhere, in the file system, in jcr, fetched by a special github provider etc. Therefore I started with implementing a resource decorator instead which was configured with a path prefix. All resources beneath that path with the extension md where "decorated" in a similar way as you describe. What are you using for the md to html transformation? Regards Carsten Robert Munteanu wrote > Hi, > > I've been toying around with a Markdown resource provider in the > whiteboard [0]. It's very much a work-in-progress but since it does > something - aka render Markdown files, including support for arbitrary > Sling properties, I would like to discuss a couple of design decisions. > > 1. The resource is created from a markdown file in the following way: > > - if a first level heading is found, place the text into jcr:title > - if a YAML front matter is found, save the properties in the resource > properties > - render everything else as HTML into jcr:description > > I am not sure that rendering the markdown in the resource provider is > the best way. On one hand, it keeps everything simple and does not > require special work in the rendering script. On the other hand, > rendering JSON/TXT with HTML inside is not that nice. > > 2. Only files with the 'md' extension are handled, all others are ignored. > > To generate static sites, usually markdown files are mixed with images, > css, js files. Does it make sense to 'pass-through' these files and > expose them as plain files? There could be some ties with the > FsResourceProvider. Alternatively the static assets can be placed in a > different subtree. > > Thoughts? > > Thanks, > > Robert > > [0]: > https://github.com/apache/sling-whiteboard/tree/master/mdresourceprovider -- Carsten Ziegeler Adobe Research Switzerland cziege...@apache.org
Markdown resource provider
Hi, I've been toying around with a Markdown resource provider in the whiteboard [0]. It's very much a work-in-progress but since it does something - aka render Markdown files, including support for arbitrary Sling properties, I would like to discuss a couple of design decisions. 1. The resource is created from a markdown file in the following way: - if a first level heading is found, place the text into jcr:title - if a YAML front matter is found, save the properties in the resource properties - render everything else as HTML into jcr:description I am not sure that rendering the markdown in the resource provider is the best way. On one hand, it keeps everything simple and does not require special work in the rendering script. On the other hand, rendering JSON/TXT with HTML inside is not that nice. 2. Only files with the 'md' extension are handled, all others are ignored. To generate static sites, usually markdown files are mixed with images, css, js files. Does it make sense to 'pass-through' these files and expose them as plain files? There could be some ties with the FsResourceProvider. Alternatively the static assets can be placed in a different subtree. Thoughts? Thanks, Robert [0]: https://github.com/apache/sling-whiteboard/tree/master/mdresourceprovider
