Erik, thank you very much.
In a few lines, here is the use case (for a standard MVC web app):
1. the user defines some template code which is stored in a view file.
Think HTML with embedded Julia code for interpolating variables, if/else,
loops, etc.
A very basic example would look like this:
<html>
<head>
<title>
%= page_title
</title>
</head>
<body>
<h1>
%= greeting
</h1>
<p>
<% if is_logged_in :>
Edit account
<: else :>
Login
<: end %>
</p>
</body>
</html>
2. there are some very simple parsing rules, such as:
a. "%=" means the line should be interpreted as Julia and outputted,
b. "<% ... %>" defines a multiline Julia block
c. everything else should be considered raw output
3. the variables used in the template have to be defined / set beforehand
by the user, usually in a controller file - and somehow be made available
to the template rendering function
---
This means that:
a. I do not know the variables beforehand, they are user inputed as part of
the template.
b. also, it's not an issue of "This is indeed one of the strengths of
Julia, and it requires neither string manipulation, nor parsing, nor
creating files nor include.". I can not avoid creating, loading and parsing
files, because that's what I want to do. I want to allow the users to
create view files that are basically HTML files with embedded Julia code.
b. I'm looking for a way to load and interpret multiline template code
stored in a file. This seems to be tricky by itself as parse(...) works
line by line (not good for me, I need to interpret the whole block at once,
for example to handle if/else blocks) and I could not find a functional
alternative for quote ... end or a way to pass a string into a quote ...
end block. Like I said, the only thing close to achieving this would be
include_string()
c. I'd like to do this in a performant manner by avoiding the dreaded
global scope (but not really by avoiding eval, which I think it's a
necessary evil in this case).
---
Strictly addressing your questions:
> If the variables are declared in a function, then there must be code that
uses them; how would this code know which variables exist? Is there e.g. a
globally known list of variables that will be provided? Or will the
variables e.g. only be used in strings to generate html code?
I go over the template file line by line and generate Julia code (as a
string). This code is either logic (if/else, loops, etc) or raw output. For
instance, the parsing of the above template would generate the following
Julia code:
"____output = Vector{AbstractString}()\npush!(____output,
\"<html>\")\npush!(____output, \" <head>\")\npush!(____output, \"
<title>\")\n____L9OHE6m05ZtELQIOMV8PPH0L = page_title\npush!(____output,
\"\$(____L9OHE6m05ZtELQIOMV8PPH0L)\")\npush!(____output, \"
</title>\")\npush!(____output, \" </head>\")\npush!(____output, \"
<body>\")\npush!(____output, \" <h1>\")\n____wHbq5V7InZ2OJKpatWu7MtUU =
greeting\npush!(____output,
\"\$(____wHbq5V7InZ2OJKpatWu7MtUU)\")\npush!(____output, \"
</h1>\")\npush!(____output, \" <p>\")\nif
is_logged_in\npush!(____output, \" Edit
account\")\nelse\npush!(____output, \"
Login\")\nend\npush!(____output, \" </p>\")\npush!(____output, \"
</body>\")\npush!(____output, \"</html>\")\npush!(____output, \"\")"
Then this code is eval'd - at this time, the variables should already be in
scope otherwise Julia won't find them.
Yes, the variables are only used to generate HTML code. So ideally they
should only be visible in the minimum scope necessary to eval this code.
> Can you elaborate on why you want to access dictionary elements as local
variables?
I was hoping this could be an approach to eval the code with its vars in a
non-global scope. Something in the lines of:
function render_tpl(output::AbstractString, vars::Dict{Symbol,Any}) #
output would be the above generated code and vars the list of vars used in
the template
# code here to expand the vars dict into function local variables
# code here to eval the output (which would be the "____output =
Vector{..." code above) -- this would automatically have access to the vars
# return the resulted ____output Vector which now contains the rendered
template
end
Thanks,
Adrian
duminică, 14 august 2016, 17:59:40 UTC+2, Erik Schnetter a scris:
>
> Adrian
>
> Can you give more details for how the variables would be used? If the
> variables are declared in a function, then there must be code that uses
> them; how would this code know which variables exist? Is there e.g. a
> globally known list of variables that will be provided? Or will the
> variables e.g. only be used in strings to generate html code?
>
> You can generate the syntax tree for function that contains the
> definitions from the dictionary, and then use `eval` to create the
> function. (Alternatively, you can use a macro or a generated function.)
> This is indeed one of the strengths of Julia, and it requires neither
> string manipulation, nor parsing, nor creating files nor include.
>
> Here is an example:
>
> fun_expr(var, val) = quote
> function f(x)
> $var = $val
> x + y
> end
> end
>
> eval(fun_expr(:y, :42))
>
> `fun_expr` creates a syntax tree (an `Expr`) that defines a function. The
> function is quoted. The function arguments `var` and `val` are inserted
> into that function, forming an assignment. Thus `var` better be an
> identifier (or something else that can be on the left of an assignment
> operator), and `val` can be an arbitrary value. Given how the function
> looks, `var` essentially has to be `y` since it's used in the next line; of
> course, this was just my arbitrary choice.
>
> The call to `eval` then passes the respective arguments, choosing the
> symbol `y` for the variable name, and the value `42` as value, and defines
> the function `f`. If you then call `f(2)`, the result is 44.
>
> Of course, you can define this function `f` only once. Julia does not
> allow changing functions. If you want to create many different functions,
> you would use anonymous functions instead.
>
> Having said this -- it's not clear that this is indeed the right way to
> address your problem; there might be a better solution. Can you elaborate
> on why you want to access dictionary elements as local variables?
>
> -erik
>
>
>
> On Sun, Aug 14, 2016 at 11:13 AM, Adrian Salceanu <adrian....@gmail.com
> <javascript:>> wrote:
>
>> Thanks
>>
>> Maybe I wasn't clear enough - otherwise, can you please elaborate, I'm
>> definitely still poking around, any clarifications would be highly
>> appreciated.
>>
>> > creating a new module
>> -> the module is available at compile time (the users of the templating
>> system will place the vars in there, by convention).
>>
>> > parse julia code
>> -> it's not really parsing julia code, it has no meaning at the point.
>> It's simply basic string processing and it's fast - tried with a 10K lines
>> HTML file, no sweat.
>>
>> > defining globals
>> -> why are they globals? include_string() is used inside a function,
>> inside a module within the app.
>>
>> > eval in a module
>> -> true, but then what can we do? That's the way of doing metaprogramming
>> in Julia, and it's widely used, isn't it?
>> I guess that would be the price for not having to do
>> print("<html><head>...") like our ancestors used to in PHP or ASP, when not
>> being chased by tigers (or something around that age).
>>
>>
>> duminică, 14 august 2016, 15:01:47 UTC+2, Yichao Yu a scris:
>>>
>>>
>>>
>>> On Sun, Aug 14, 2016 at 6:13 PM, Adrian Salceanu <adrian....@gmail.com>
>>> wrote:
>>>
>>>> Variables contained in a module and then parsed Julia code included
>>>> within a function using include_string().
>>>>
>>>> Any obvious performance issues with this approach?
>>>>
>>>
>>> Everything about it?
>>> Literally every steps are hitting the slow path that is only meant to
>>> execute at compile time and not runtime. Including
>>>
>>> Creating a new module, parse julia code, eval in a module, defining
>>> globals.
>>>
>>>
>>>>
>>>>
>>>> duminică, 14 august 2016, 12:11:06 UTC+2, Adrian Salceanu a scris:
>>>>>
>>>>> OK, actually, that's not nearly half as bad. Variables contained in a
>>>>> module
>>>>>
>>>>> include("src/Ejl_str.jl")
>>>>> using Ejl
>>>>>
>>>>>
>>>>> module _
>>>>> couñtry = "España"
>>>>> lang = "en"
>>>>> end
>>>>>
>>>>>
>>>>> function render_template()
>>>>> tpl_data = ejl"""
>>>>> <% if _.lang == "en" :>
>>>>> Hello from me, ...
>>>>> <: else :>
>>>>> Hola
>>>>> <: end %>
>>>>>
>>>>>
>>>>> %= _.couñtry == "España" ? "Olé" : "Aye"
>>>>> moo
>>>>> """
>>>>>
>>>>>
>>>>> include_string(join(tpl_data, "\n"))
>>>>> join(____output, "\n")
>>>>> end
>>>>>
>>>>>
>>>>> render_template() |> println
>>>>>
>>>>> Hello from me, ...
>>>>>
>>>>>
>>>>> Olé
>>>>> moo
>>>>>
>>>>>
>>>>> duminică, 14 august 2016, 11:20:37 UTC+2, Adrian Salceanu a scris:
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Yes, I've thought about a few ways to mitigate some of these issues:
>>>>>>
>>>>>> 1. in the app I can setup a module (like Render) and evaluate into
>>>>>> this module exclusively.
>>>>>> Hence, another approach might be to have some helper methods that
>>>>>> setup the variables in the module and then eval the template itself
>>>>>> inside
>>>>>> the module too (must try though). So something in the lines of:
>>>>>> set(:foo, "foo")
>>>>>> set(:bar, [1, 2, 3])
>>>>>> parse_tpl(ejl"""$(foo) and $(bar)""")
>>>>>> # all the above gets parsed in Render
>>>>>>
>>>>>> 2. break the parsing in 2 steps:
>>>>>> a. reading the template string and parse it to generated Julia code
>>>>>> (as strings) (an array of Julia code lines) - cache it
>>>>>> b. (load strings from cache and) eval the code together with the vars
>>>>>>
>>>>>> ===
>>>>>>
>>>>>> Another approach (which is how it's done in one of the Ruby
>>>>>> templating engine) is to generate a full function definition, whose body
>>>>>> parses the template and takes the variables as params. And then eval and
>>>>>> execute the function with its params. However, I'm still struggling with
>>>>>> the metaprogramming API as for instance parse() chokes on multiple
>>>>>> lines,
>>>>>> and I couldn't find a functional equivalent of a quote ... end blocks.
>>>>>> But
>>>>>> I'm hoping include_string() will do the trick (must test though).
>>>>>>
>>>>>>
>>>>>> sâmbătă, 13 august 2016, 15:20:01 UTC+2, Yichao Yu a scris:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Aug 13, 2016 at 8:06 PM, Adrian Salceanu <
>>>>>>> adrian....@gmail.com> wrote:
>>>>>>>
>>>>>>>> That's pretty difficult as my goal is to use embedded Julia as the
>>>>>>>> templating language. Similar to Ruby's ERB, ex:
>>>>>>>> http://www.stuartellis.eu/articles/erb/
>>>>>>>>
>>>>>>>> So say in the template I have something like
>>>>>>>>
>>>>>>>> <% if foo == "bar" %>
>>>>>>>> Bar
>>>>>>>> <% else %>
>>>>>>>> Baz
>>>>>>>> <% end %>
>>>>>>>>
>>>>>>>> The idea is to use Julia itself to parse the code block and Julia
>>>>>>>> will raise an error is foo is not defined. So I can't really look it
>>>>>>>> up.
>>>>>>>>
>>>>>>>
>>>>>>> It's ok to use the julia syntax and parser but it's a pretty bad
>>>>>>> idea to use the julia runtime to actually evaluating the expression,
>>>>>>> and
>>>>>>> absolutely not by making them reference to local variables.
>>>>>>>
>>>>>>> For a start you are not allowed to reference local variables by
>>>>>>> names anyway.
>>>>>>> You also shouldn't allow reference to/overwrite of other local
>>>>>>> variables (i.e. the template namespace should be fully isolated and
>>>>>>> independent of any scope in the template engine).
>>>>>>>
>>>>>>> Since you want to eval, it seems that efficiency is not an issue, in
>>>>>>> which case you can create an anonymous module and eval/create globals
>>>>>>> in
>>>>>>> that module. This should also be reasonably fast if you are only using
>>>>>>> the
>>>>>>> template once.
>>>>>>>
>>>>>>> If you want to use it multiple time and compile the template, you
>>>>>>> should then scan for variable references in the expressions and process
>>>>>>> it
>>>>>>> from there.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> I can either do
>>>>>>>>
>>>>>>>> <% if _[:foo] == "bar" %>
>>>>>>>>
>>>>>>>> or
>>>>>>>>
>>>>>>>> <% if _(:foo) == "bar" %>
>>>>>>>>
>>>>>>>> but it's not that nice.
>>>>>>>>
>>>>>>>>
>>>>>>>> sâmbătă, 13 august 2016, 13:24:18 UTC+2, Yichao Yu a scris:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Aug 13, 2016 at 7:13 PM, Adrian Salceanu <
>>>>>>>>> adrian....@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> It's for a templating engine. The user creates the document (a
>>>>>>>>>> string) which contains interpolated variables placeholders and
>>>>>>>>>> markup. When
>>>>>>>>>> the template is rendered, the placeholders must be replaced with the
>>>>>>>>>> corresponding values from the dict.
>>>>>>>>>>
>>>>>>>>>> The lines in the template are eval-ed and so Julia will look for
>>>>>>>>>> the variables in the scope. So the vars should be already defined.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> You should explicitly look up those variables in the dict instead.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, ultimately I can force the user to use a dict (or rather a
>>>>>>>>>> function for a bit of semantic sugar) - which is preferable from a
>>>>>>>>>> performance perspective, but less pretty end error prone from the
>>>>>>>>>> user
>>>>>>>>>> perspective.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>
>
>
> --
> Erik Schnetter <schn...@gmail.com <javascript:>>
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
