Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-20 Thread Daniel Kinzler
Am 19.05.2014 23:05, schrieb Gabriel Wicke: I think we have agreement that some kind of tag is still needed. The main point still under discussion is on which tag to use, and how to implement this tag in the parser. Indeed. Originally, domparse was conceived to be used in actual page content

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-20 Thread Gabriel Wicke
On 05/20/2014 02:46 AM, Daniel Kinzler wrote: My main reason for recycling the html tag was to not introduce a new tag extension. domparse may occur verbatim in existing wikitext, and would break when the tag is introduces. The only existing mentions of this are probably us discussing it ;) In

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Daniel Kinzler
I'm getting the impression there is a fundamental misunderstanding here. Am 18.05.2014 04:28, schrieb Subramanya Sastry: So, consider this wikitext for page P. == Foo == {{wikitext-transclusion}} *a1 map .. ... /map *a2 {{T}} (the html-content-model-transclusion) *a3 Parsoid

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Subramanya Sastry
On 05/19/2014 04:52 AM, Daniel Kinzler wrote: I'm getting the impression there is a fundamental misunderstanding here. You are correct. I completely misunderstood what you said in your last response about expandtemplates. So, the rest of my response to your last email is irrelevant ... and

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Daniel Kinzler
Am 19.05.2014 14:21, schrieb Subramanya Sastry: On 05/19/2014 04:52 AM, Daniel Kinzler wrote: I'm getting the impression there is a fundamental misunderstanding here. You are correct. I completely misunderstood what you said in your last response about expandtemplates. So, the rest of my

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Gabriel Wicke
On 05/19/2014 09:52 AM, Daniel Kinzler wrote: Am 18.05.2014 16:29, schrieb Gabriel Wicke: The difference between wrapper and property is actually that using inline wrappers in the returned wikitext would force us to escape similar wrappers from normal template content to avoid opening a gaping

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Gabriel Wicke
On 05/19/2014 04:54 PM, Gabriel Wicke wrote: The move to HTML-based (self-contained) transclusions expansions will avoid this issue completely. That's a few months out though. Maybe we can find a stop-gap solution that moves in that direction, without introducing special tags in

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Gabriel Wicke
On 05/19/2014 10:19 AM, Gabriel Wicke wrote: On 05/19/2014 04:54 PM, Gabriel Wicke wrote: The move to HTML-based (self-contained) transclusions expansions will avoid this issue completely. That's a few months out though. Maybe we can find a stop-gap solution that moves in that direction,

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Gabriel Wicke
On 05/19/2014 10:55 AM, Bartosz Dziewoński wrote: I am kind of lost in this discussion, but let me just ask one question. Won't all of the proposed solutions, other than the one of just not expanding transclusions that can't be expanded to wikitext, break the original and primary purpose of

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Bartosz Dziewoński
I am kind of lost in this discussion, but let me just ask one question. Won't all of the proposed solutions, other than the one of just not expanding transclusions that can't be expanded to wikitext, break the original and primary purpose of ExpandTemplates: providing valid parsable wikitext,

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Daniel Kinzler
Am 19.05.2014 20:01, schrieb Gabriel Wicke: On 05/19/2014 10:55 AM, Bartosz Dziewoński wrote: I am kind of lost in this discussion, but let me just ask one question. Won't all of the proposed solutions, other than the one of just not expanding transclusions that can't be expanded to wikitext,

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-19 Thread Gabriel Wicke
On 05/19/2014 12:46 PM, Daniel Kinzler wrote: Am 19.05.2014 20:01, schrieb Gabriel Wicke: On 05/19/2014 10:55 AM, Bartosz Dziewoński wrote: I am kind of lost in this discussion, but let me just ask one question. Won't all of the proposed solutions, other than the one of just not expanding

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-18 Thread Gabriel Wicke
On 05/18/2014 02:28 AM, Subramanya Sastry wrote: However, in his previous message, Gabriel indicated that a property in the JSON/XML response structure might work better for multi-part responses. The difference between wrapper and property is actually that using inline wrappers in the returned

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-17 Thread Daniel Kinzler
Am 16.05.2014 21:07, schrieb Gabriel Wicke: On 05/15/2014 04:42 PM, Daniel Kinzler wrote: The one thing that will not work on wikis with $wgRawHtml disabled is parsing the output of expandtemplates. Yes, which means that it won't work with Parsoid, Flow, VE and other users. And it has been

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-17 Thread Subramanya Sastry
(Top posting to quickly summarize what I gathered from the discussion and what would be required for Parsoid to expand pages with these transclusions). Parsoid currently relies on the mediawiki API to preprocess transclusions and return wikitext (uses action=expandtemplates for this) which

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-17 Thread Subramanya Sastry
On 05/17/2014 10:51 AM, Subramanya Sastry wrote: So, going back to your original implementation, here are at least 3 ways I see this working: 2. action=expandtemplates returns a html.../html for the expansion of {{T}}, but also provides an additional API response header that tells Parsoid

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-17 Thread Gabriel Wicke
On 05/17/2014 05:57 PM, Subramanya Sastry wrote: On 05/17/2014 10:51 AM, Subramanya Sastry wrote: So, going back to your original implementation, here are at least 3 ways I see this working: 2. action=expandtemplates returns a html.../html for the expansion of {{T}}, but also provides an

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-17 Thread Daniel Kinzler
Am 17.05.2014 17:57, schrieb Subramanya Sastry: On 05/17/2014 10:51 AM, Subramanya Sastry wrote: So, going back to your original implementation, here are at least 3 ways I see this working: 2. action=expandtemplates returns a html.../html for the expansion of {{T}}, but also provides an

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-17 Thread Subramanya Sastry
On 05/17/2014 06:14 PM, Daniel Kinzler wrote: Am 17.05.2014 17:57, schrieb Subramanya Sastry: On 05/17/2014 10:51 AM, Subramanya Sastry wrote: So, going back to your original implementation, here are at least 3 ways I see this working: 2. action=expandtemplates returns a html.../html for the

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-16 Thread Daniel Kinzler
Hi again! I have rewritten the patch that enabled HTML based transclusion: https://gerrit.wikimedia.org/r/#/c/132710/ I tried to address the concerns raised about my previous attempt, namely, how HTML based transclusion is handled in expandtemplates, and how page meta data such as resource

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-16 Thread Gabriel Wicke
On 05/15/2014 04:42 PM, Daniel Kinzler wrote: The one thing that will not work on wikis with $wgRawHtml disabled is parsing the output of expandtemplates. Yes, which means that it won't work with Parsoid, Flow, VE and other users. I do think that we can do better, and I pointed out possible

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-15 Thread Daniel Kinzler
Am 14.05.2014 16:04, schrieb Gabriel Wicke: On 05/14/2014 03:22 PM, Daniel Kinzler wrote: My patch doesn't change the handling of html.../html by the parser. As before, the parser will pass HTML code in html.../html through only if wgRawHtml is enabled, and will mangle/sanitize it otherwise.

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-14 Thread Gabriel Wicke
On 05/13/2014 05:37 PM, Daniel Kinzler wrote: Hi all! During the hackathon, I worked on a patch that would make it possible for non-textual content to be included on wikitext pages using the template syntax. The idea is that if we have a content handler that e.g. generates awesome

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-14 Thread Daniel Kinzler
Thanks all for the imput! Am 14.05.2014 10:17, schrieb Gabriel Wicke: On 05/13/2014 05:37 PM, Daniel Kinzler wrote: It sounds like this won't work well with current Parsoid. We are using action=expandtemplates for the preprocessing of transclusions, and then parse the contents using Parsoid.

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-14 Thread Gabriel Wicke
On 05/14/2014 01:40 PM, Daniel Kinzler wrote: This means that HTML returned from the preprocessor needs to be valid in wikitext to avoid being stripped out by the sanitizer. Maybe that's actually possible, but my impression is that you are shooting for something that's closer to the behavior

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-14 Thread Daniel Kinzler
Am 14.05.2014 15:11, schrieb Gabriel Wicke: On 05/14/2014 01:40 PM, Daniel Kinzler wrote: This means that HTML returned from the preprocessor needs to be valid in wikitext to avoid being stripped out by the sanitizer. Maybe that's actually possible, but my impression is that you are shooting

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-14 Thread Gabriel Wicke
On 05/14/2014 03:22 PM, Daniel Kinzler wrote: My patch doesn't change the handling of html.../html by the parser. As before, the parser will pass HTML code in html.../html through only if wgRawHtml is enabled, and will mangle/sanitize it otherwise. Oh, I thought that you wanted to support

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-14 Thread Dan Andreescu
Can you outline how RL modules would be handled in the transclusion scenario? The current patch does not really address that problem, I'm afraid. I can think of two solutions: * Create an SyntheticHtmlContent class that would hold meta info about modules etc, just like ParserOutput -

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-13 Thread Brad Jorsch (Anomie)
On Tue, May 13, 2014 at 11:37 AM, Daniel Kinzler dan...@brightbyte.dewrote: As Brion pointed out in a comment to my original, there is another caveat: what should the expandtemplates module do when expanding non-wikitext templates? I decided to just wrap the HTML in html.../html tags instead

Re: [Wikitech-l] Transcluding non-text content as HTML on wikitext pages

2014-05-13 Thread Matthew Flaschen
On 05/13/2014 11:37 AM, Daniel Kinzler wrote: Hi all! During the hackathon, I worked on a patch that would make it possible for non-textual content to be included on wikitext pages using the template syntax. The idea is that if we have a content handler that e.g. generates awesome diagrams from