Re: Template Engine Compilation and Runtime Refactoring Progress
Hi, Current status: The freezing of free variables into a context dictionary for includes and extension is currently not working as expected. The reason is obvious but the solution is not entirely clear to me yet. I need to track what variables were assigned to until a specific statement, not only what variables will be assigned or referenced for a block. I don't like to do that while transforming the AST, but walking that another time sounds like a horrible idea was well. A bit torn, I might just use the current identifier tracking code and also record the id of the node that triggers the assignment. Then I could reconstruct the state before each statement I think. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: [GSOC] Template Engine Compilation and Runtime Refactoring Progress
Hi, Status of this week: I spend the last week mostly trying to figure out how to debug a bunch of annoying issues and due to the slow progress I decided on doing an AST -> Python code translation for debugging purposes. The identifier tracking method I naively started using initially (which is assuming that there are no branches and recording identifiers for each scope while the transformer is creating code) turns out to be by far not good enough as it misses identifiers that are referenced after blocks. I will need to heavily improve on that. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To view this discussion on the web visit https://groups.google.com/d/msg/django-developers/-/mITAVQia2fEJ. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Template Engine Compilation and Runtime Refactoring Progress
Hi, Progress update. This week I was starting the work on the compiler and I have at least for Django's use case most compilation things ironed out. However it turns out that compiling AST nodes made from scratch is a painful process as the interpreter will randomly segfault or fail compiling with useless error messages. For debugging the AST I wrote a bunch of small helpers but the current HEAD revision exhibits some weird behavior I have not found the cause for yet. Generally however good progress for a week full of EuroPython :) Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Template Engine Compilation and Runtime Refactoring Progress
Hi, On Jun 17, 2:18 pm, Anatoly Vostryakov wrote: > I've thought today. May be it will possible to add support of > parameters for functions or object methods in django templates. > I mean, something like this in django template: The compiler is able to handle this case, but the syntax of the Django templating language does not. I will not change any features in the Django template engine during this proposal and actually reused most of the parser toolchain until fall. If someone wants to step up and continue improving the features on the language after GSOC I am happy to assist. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Template Engine Compilation and Runtime Refactoring Progress
Hi, I'm in between two conferences right now and university and a bunch of other stuff needs attention as well so progress is slowed down quite a bit right now. Will however catch up next week with what I cannot do this one. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Template inheritance and duplicated content.
Hi, On May 24, 10:58 am, Jonathan Slenders wrote: > I guess this is a flaw in the inheritance algorithm, only the most > outer blocks should be used during the resolving of inheritance. How else would it work? Think of a block like a function plus the execution of that function. By placing a new block within a block you want that behavior and if you override a block and in that another block that already existed both Jinja2 and Django will have to also immediately call it. There is only one rule: if you don't like that, make the block toplevel. The only alternative sane semantic would be raising a runtime error, but I don't think that is a good idea. Regards, That Jinja guy -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Template Engine Compilation and Runtime Refactoring Progress
Hi, Weekly progress report: I started on documenting some of the behavior that the current django template engine has and how it affects the compilation. The nodes are mostly implemented and some of them can be interpreted already. I want to finish the interpreter first because that way we can easily figure out if it's compatible with the existing interpreter and if it is, start on compiling it down to AST nodes and from that into actual Python bytecode. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Template Engine Compilation and Runtime Refactoring Progress
Hi, The progress went slower than expected because I had to analyze some more runtime differences between Django and Jinja2 which I missed at first. The main problem are changes in the safe-string behavior but I think I found a possible solution to counter that. I had to diverge from my previous plan to keep the nodes unchanged from Jinja2 so at that point it's pretty sure that Jinja2 will not survive this GSOC without breaking backwards compat on it's extension interface either which now simplify further plannings. First pieces of currently meaningless code are checked in, so is an OBSERVATIONS file which has some notes on internals. I will update this file with more internal notes as the project progresses. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Template Engine Compilation and Runtime Refactoring Progress
Hi, The repository with my changes will be here: https://github.com/mitsuhiko/django-template-compilation As this proposal implements a 'newtemplates' module in Django instead of modifying the existing one until it's proven to be compatible in every single regard, I will have it as a separate repository. The idea is that it can then easily be tested with other changes in Django itself. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
[GSOC] Template Engine Compilation and Runtime Refactoring Progress
Hi, This is the first weekly status update of many to come. Just a quick overview of this project and myself. My nickname on freenode is 'mitsuhiko' and this is the preferred form to reach me in general. Alternatively my twitter account with the same name is a good way of reaching me even when I'm not at my computer, so is mail. As the template system in Django is already a pretty isolated system by itself it shouldn't be too hard to keep the work nicely merge-able. I have not yet decided if I will keep this branch on github or somewhere else. The Jinja2/Django maintenance aspect of this will be decided at a late point, probably even just after GSOC so it can be ignored for the duration of GSOC. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposal for Template Engine Refactoring
Hi, On Mar 29, 3:54 pm, Jacob Kaplan-Moss wrote: > Can you speak a little more about how you'd see the workflow working > with this new library? It seems a bit complex to have a library that's > *copied* between Django and Jinja: I'd worry about patches getting > lost. I think that problem can mostly be avoided. The interpreter / compiler / ast combo library is probably very minimal in size. I don't see a big problem having to copy them into the Django svn every once in a while. That could even be automated. > For context, this has happened a few times with the simplejson and > elementtree libraries in core Python: patches made to the upstream got > "lost" before merging into Python's stdlib, and patches made against > the stdlib didn't make it upstream. stdlib is a more complex problem because it's tightly tied into the Python release cycle and has a separate set of developers. Even if I'm not a django developer I am never far from the project so it will be the same developer doing it during GSOC and afterwards. The extra work involved in making this work with both Jinja2 and Django and merging every once in a while should be minimal. The only problem would be other people doing changes to the codebase directly in the Django SVN. Which is why I would recommend having a dedicated separate repository for this thing (might as well be subversion if you want) where I and all Django developers have access for bugfixes if necessary. And also, once that thing is stable I don't think there is a lot to be done on the actual compiler. The Django specific stuff will not end up in that backend anyways. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposal for Template Engine Refactoring
Hi, On Mar 29, 2:23 pm, Jonathan S wrote: > Are you planning on keeping the API for registering template tags > compatible? (django.template.Library.register.tag) It will be supported because otherwise upgrading won't be possible, but how well this works in detail would have to be investigated and what a cleaner implementation looks like. Jinja2 has an expression parser and encourages people to create a AST that does what they want in extensions. However in Jinja2 I never encouraged people to write custom tags so the API is very limited there. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Proposal for Template Engine Refactoring
Great. Google groups decided to reflow my mail. Here in actually readable version: http://paste.pocoo.org/show/362024/ Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Proposal for Template Engine Refactoring
Hi, I would like to participate in this year's Google Summer of Code by introducing a new backend for the Django (and Jinja2) templating language. As the author of Jinja2 I recently decided to rewrite the code generation of the engine to better support alternative Python implementations like PyPy and restricted environments like Google's Appengine. The Jinja2 backend is already usable for other templating engines as well for as long as the sematics are compatible with the ones from Jinja2. However with a better backend design the Django semantics should be possible to implement in there as well. As such as I am proposing to keep this backend independent of either Django and Jinja2. I would maintain inside a separate repository and with each release of Jinja2 and Django port fixes into their respective code bases. This could also be used as a basis for other template engines or domain specific languages if necessary. The reason I don't want to keep this a separate package but actually copy the code over to Django and Jinja2 are twofold. First of all I don't want to introduce a new dependency for the template engines themselves. Secondly I really want to keep this an implementation detail for Jinja2 at the very least. In Django we don't want to see dependencies either. Reasons (coming from the current Jinja2 backend) for a new version: 1. implement a drop-in interpreter as alternative to the template compilation for better error reporting during development. 2. implement an ast.py based compiler in order to avoid the debug traceback hackery that is currently in place to support debugging of templates with accurate line numbers. 3. Better support for jit interpreted Python installations like PyPy by removing the need to call locals() or sys._getframe which is currently required by Jinja2 in order to implement includes. 4. Python generating compiled for GAE. This one would not have the same error handling capabilities as the interpreter and the AST based compiler, but has the advantage of running everywhere and would allow module based importing like the one currently in use in Jinja2. Reasons why this is interesting for Django: 1. It would greatly speed up execution of core tags. The idea is to use code generation for the builtin tags and to generate a context as necessary when droppping into old-style custom tags. 2. Provide a new API that can be used for new tags that allow code generation behind the scenes. 3. Better debugging capabilties by providing actual Python tracebacks for errors in templates. How much speedup will we see in Django? Probably not as much as people are hoping as Django's current semantics just cannot be sped up to Jinja2 levels by keeping the extension API unchanged. However because there are already some abstractions in place we can probably achieve a significant speedup for certain common template types that it reduces the need to switch to Jinja2. Why am I proposing this as a GSOC Django project instead of hacking on this under the umbrella of the PSF for Jinja2 alone? First of all, speeding up Jinja2 is a boring task right now as it is very close to raw Python performance already. Secondly there was a proposal by Alex last year and there is already a proposal here about template engine compilation backends and if I am going to do this for Jinja2 anyways, I can keep it generic enough to also support Django. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
#10216 - TEMPLATE_DEBUG / TemplateSyntaxError handling doesn't play nice with Jinja2
Hello everybody, I would love to propose a new solution for ticket #10216 which is currently a wontfix. Basically the problem is that with TEMPLATE_DEBUG turned on, Django expects a certain protocol from exceptions catched by the debugging system: Either an exception does not have a .source attribute, or it has one and this is a tuple that can be unpacked where one of the items is an template loader origin and the second is the position in the template, as far as I can see that. Now the direct problem for me is that people get no proper debug output in Django if Jinja 2 raise an exception, because the .source attribute of a Jinja 2 exception has a different protocol: it's a string. Of course I can work around that problem in Jinja and tell people to disable TEMPLATE_DEBUG, but I think the issue is not limited to Jinja 2. The purpose of a debugging middleware is to show what went wrong. Currently the debugging middleware does the opposite if you have any exception with a source attribute that is not a template origin and position information. A quick search on google code showed me that .source is not an uncommon attribute on exceptions. For example PyXML uses it in the xpath for SyntaxException and internal exceptions. If any of these would be raised in a django application, the user would experience a broken debug view. Not sure how common PyQwt is in web applications but that would have the same problem ;) I would like to propose a different solution: do an instance check against a django TemplateSyntaxError (or any other template exception that has these .source attributes). That should be easy to accomplish and the debug system already has a dependency on the template engine anyways for the rendering of the UI. Regards, Armin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: r9766-related issues
Hi, On 13 Mai, 00:42, Yuri Baburov wrote: > Also on OS X you can set if filename is case-sensitive on per-volume > basis, when formatting, and usually it's case-insensitive. windows is > always case insensitive, linux is usually case-sensitive. I'm not talking about case sensitivity here but unicode normalization which is controlled in the IO system. For example if you're creating a file öäü.txt (\x94\x84\x81) OS X will store it as o\u0308a\u0308u \u0308.txt on the filesystem. No matter if the filesystem is HSF, NFS, a mounted samba share or anything else. On Linux the filesystem might have an encoding but is generally non unicode-aware and no normalization takes place at all. This could lead to weird effects where you have \x94\x84\x81.txt in the database but o\u0308a\u0308u \u0308.txt on the filesystem which obviously makes it impossible to move the data from an OS X system to a linux one or access an NFS share from two systems at the same time. > This does matter when one filename has the same letters as another but > some letters have different case. This also can cause troubles when > moving existing DB and files from one filesystem to another. I'm well aware of that, but that's a different story. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: r9766-related issues
Hi, On 11 Mai, 15:03, Marty Alchin wrote: > If you're using the filename to store it somewhere else, typically for > denormalization, it'd be better to do that post-save, since then you > know the record actually got saved in the database. Otherwise, you > might be trying to access the content of the file, which would be > better using the direct file access, which can often save yourself a > round-trip to the storage backend (which can be a big win for paid > storage solutions, like S3). I was reading about #8593 today and I think this adds a new layer of complexity to the problem. Some operating systems perform filename normalization on their own. While the case covered in the bug only affects Windows machines and is a weird behavior in the Python ntpath module instead of the Kernel, it could cause some troubles. On an OS X machine the operating system will normalize the filename to some sort of unicode normalization form. (The exact one depends on the version of OS X you're running). This translation happens in a way that you will not notice it from inside your application. The normalization however happens in the IO system and not the HSF file system. Now imagine you're writing to an NFS share that is also accessed from linux. If they two would use the same database as well, the Linux version would not be able to locate the file on the filesystem because the entry in the database would be only valid if the OS does the normalization again when opening the file. This sounds unrelated to the bug we're discussing here, but it raises the problem that the value that ends up in the database is probably not exactly the value the real name of the file. Maybe there should be a setting in django to switch the filename generation for the uploaded files to something pre-normalized inside django to make the filename more reliable. (Maybe go as far as lowercase and ASCII only?). Either way it shows that the filename on the file system is unreliable for pretty much everything except loading and replacing the file. On file creation the code can already "rename" the file to avoid clashes. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: r9766-related issues
Hi, All bugs are fixed now except for #10788. Now the problem with closing that one is that this one requires a design descision. I updated the ticket accordingly for jacob or anyone else to decide on it. My personal opinion is that I consider it bad design for the application to depend on the final filename of an uploaded file on the filesystem. This makes it nearly impossible to create "atomic" uploads for unique filenames. Regards, Armin Links: http://code.djangoproject.com/ticket/10788 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: r9766-related issues
Heyho, On May 7, 10:01 pm, Marty Alchin wrote: > It's not explicitly related to the MRO and method stuff you and Alex > have been working on, but it's definitely related to the r9766 > discussion, since it's caused by the delayed saving. I have a clear > understanding of the problem, but I don't have time to write it up at > the moment, so I'll try to catch you guys on IRC if you're still up > when I get home. Florian Apolloner just wrote a patch for that and attached it to the ticket. > That only solves the problem if you're trying to access the file's > contents, which I'll admit is the majority case. Other apps may need > the filename for other reasons, though, and it was available in 1.0, > so the current behavior is backwards-incompatible. Of course, all the > uses I can think of would actually be better as post_save handlers > anyway, but the fact is that compatibility was broken, so we should > see if we can restore it. There are not many situations where you would need the filename of the thing in the pre-save command. If we can collect some situations where this would be useful it makes sense to think about it. I personally think it was a design mistake in the first place and should be changed to the behavior it's showing currently. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: r9766-related issues
Hi, On May 7, 5:37 pm, Karen Tracey wrote: > #10249: can't create consistent MRO (method resolution order) when assigning > a File to a FileField. This is fixed. > #10300: custom storage backend can't get length of content to save. This *should* be fixed. I can't test it, no access to S3. > #10404: height_field and width_field for an ImageField don't always work. This is not really related to that and I have a hard time understanding that part of the ORM, but it should be fixable. > #10788: actual file name not known until after model save. That's the problem with the new implementation. The final filename is decided when the file is actually saved. The cleaner way is to use the assigned file and directly read from it when it's still in the temporary location or memory. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: r9766-related issues
Hi, On May 7, 5:37 pm, Karen Tracey wrote: > So far as I know there 4 open ticks remaining related to r9766. Three are > regressions so I believe something really needs to be done about them before > 1.1; one I think is just a bug in the new function. Personally I'd rather > not revert the new function at this point (it's been available in the > alpha/beta releases, it would be nasty to pull it at this point). However, > reverting the part of r9766 that delayed saving the file to the backend > until model save (in the case where a ModelForm is used to create a model) > may be necessary to fix the last of the four problems, unless we want to > document that one as a backwards-incompatible change. I'm working with Alex on that right now here in Prague. We have some branches on github related to that. Basically the idea is to start with getting rid of some of the over engineering in the abstract base classes and make sure the subclasses call the constructors properly. Once that works we should be able to fix all the "method missing on temporary file" tickets that are currently just worked around. The fixing of the problems caused by the changeset will be covered by that branch as well. > The four tickets I know of are: > > #10249: > #10300: > #10404: > #10788: We're working on that. We'll report back some status when we get the existing testcases passing again :) Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Concrete django.template Suggestions
Hi everybody, After my quite controversal blog post about Jinja and Django I summed up some suggestions for the Django template engine that don't require a complete reimplementation / breaking backwards compatibility :) First of all the thread in-safeties I discovered [and the explanation why nobody noticed so far]. First of all the explanation, because that's probably more interesting: Per default templates are parsed and evaluated each request so you will never notice that. However I've seen enough applications by now that keep the template objects in memory where these problems become visible. Why would anyone want to not parse the templates each request? Because it's a lot faster. How much? ~100 req/sec versus 60 req/sec for the file browser page on bitbucket, and that's the overall request time including all of the request handling, http header parsing and of course database. Where are threading bugs? Everywhere, where a tag render method somehow modifies the node object. Where does this happen? From what I've seen so far it's done in "block", "ifchanged", "cycle" and "extends". Now, of course the attribute assignments happen for a reason: they are necessary, at least for cycle and ifchanged. Block and extends look like accidents, they could be solved a lot cleaner (block would put a reference to a hypothetical SuperBlock into the context, rather than a reference to itself) and extends doesn't have to store the evaluated value on the node at all, a local variable is just fine. Ifchanged and cycle however have to somewhere store the information, and I recommend using the context object for that. The context could get `get_state_var`, `set_state_var` functions that can be used to store node related information from one render call to the next:: def get_state_var(self, node, name, default=None): storage = self._state_vars.get(id(node), None) if storage is not None and name in storage: return storage[name] return default def set_state_var(self, node, name, value): self._state_vars.setdefault(id(node), {})[name] = value The cycle node could then look like this:: class CycleNode(Node): def __init__(self, cyclevars, variable_name=None): self.cycle_vars = map(Variable, cyclevars) self.variable_name = variable_name def render(self, context): pos = context.get_state_var(self, 'pos', 0) value = self.cycle_iter[pos].resolve(context) context.set_state_var(self, 'pos', (pos + 1) % len(self.cycle_vars)) if self.variable_name: context[self.variable_name] = value return value This used in all core tags makes the `Template` object thread safe, but not the `Context` which is perfectly okay because it doesn't happen (or should not happen) that a context object is shared between threads. At least I don't see a situation where this is useful. What's harder to fix is how the i18n integration translates filter arguments or gettext constants (those _("foo") thingies). Currently that happens very often at node/object creation rather than node/ object evaluation. A good example for that problem is FilterExpression.__init__. The translation would have to be moved to the resolve method in that case. When would a language change between template compilation and rendering? If the language changes each request which is a very common case on multilingual websites. Changing the parser to a more advanced system or making the template engine more consistent is not so important for the time being, but I want to raise a few situations I encountered where the behaviour is confusing: - cycle arguments are delimited by spaces and each item can be an expression (however if there is a comma involved somewhere, it seems like the tag is interpreted as comma separated list of strings which makes the 'cycle "foo", "bar", "baz"' yield unexpected results. - On the other hand arguments for the url tag are comma delimited, but whitespace after comma is not allowed. - The group-by part of the regroup tag is an expression, but everything else but a variable name returns unexpected results. Furthermore does this tag only work on dicts. By the group-by part I'm refering to the expression after the "by" keyword: {% regroupy foo by bar as blah %} bar is here treated as constant string, even though it's represented as variable in the regroup node and in the syntax. - Likewise ssi excepts one argument that looks like a variable, but is treated as constant string. Tags to be excluded from the variable-as-string rule should be block and load because they represent neither variables nor strings. The argument for block works like a label and the argument for load is a library. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to
Re: RFC: django.template refactoring (#7806)
On Sep 18, 12:52 am, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Wed, 2008-09-17 at 14:42 +0100, Ben Ford wrote: > Parts of it are very well thought out and if it had been a post on "how > Jinja works" it would have been excellent. Other parts are completely > unconstrained by facts or acknowledgement that Django's standard > templates and Jinja's ones have different goals (which is important, > since this isn't an apples to apples comparison at all). I'm starting to feel like Don Quixote here (the Windmills being a community that learned to live with the implications of the limitations of their beloved template engine). After my blog post I continued to dig in the Django sources and had to notice that it's impossible to reused Template objects at all so you are pretty much forced to reparse them over and over again. (Not reparsing would for example also break the FilterExpression embedded i18n support because it translates on compilation, not on resolving) With that in mind you can pretty much everything said about thread safety in my blog post. The rest of my reply is missing now. I actually wrote a mail here that was a lot longer than my blog post from yesterday but noticed that I repeated myself with slightly different examples. I don't want to start the discussions again because I know the first reply will be again that missing support for expressions is good, that the AST evaluator is a blessing and in general the Django template engine is the best ever happened to mankind. I know I'm sort of exaggerating here but in general that's what I read out of the comments I've seen on the reddit discussion thread about that topic. Please apologise my blog post, it wasn't meant to criticise the Django template engine concept in any way, just to inspire a possible improved version of Django templates. I think Jinja does solve a few things better than Django (and probably some worse) but I can't be a mistake to look at how things work there to get an inspiration of how to make things better. The primary inspiration for myself for Jinja was Django, and I'm not ashamed to say that. When I started developing Jinja it was a simple port of the concept into a standalone library not depending on Django with unicode support. What you can see now in Jinja2 is the second iteration of a vastly changed template engine, that however is still based on the same principles the Django template engine is: no XML, template designer friendly, easy to get started, extensible. If I can help out in any way to get some of the Jinja concepts ported to Django, or just explain why things work that way and not different in detail I would love to do so. I think Django can't lose, it just takes someone to have a look at it. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Escaping in templates...
Hoi, Another small notice. Pylons and other frameworks thought about implementing "__html__" for objects that return an html representation of the object. If there is none it's converted to unicode and escaped. Adding something like "__html__ = lambda s: s" to the escaped string base classes could improve support for other template engines like genshi. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #3527 - better debug traceback with code executing...
Hoi, A possible solution would be moving the debug view from the core into a middleware. Then it's possible to replace the middleware and move the debugger into a contrib package. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Escaping in templates...
Hi, On Apr 16, 1:08 pm, Michael Radziej <[EMAIL PROTECTED]> wrote: > Armin, you're coming a bit late to this discussion. We had a few quite > extensive threads months ago. If you seriously want to engage in this > discussion, please read what's been written before. I understand that you > cannot know when you joined later, but I wouldn't like to go through all the > arguments again and again. Dammit, haven't investigated further. Sorry for that. > - there's a way to mark strings as "don't need further escaping" > (mark_safe()) > > Whether it is an advantage or not is probably a matter of how you develop and > how much you trust your web designer (or yourself). Different people will > see different trade-offs here, that's just normal. Yeah. After reading the current implementation it looks quite sane to me. As long as there is a way to turn it off completely I can live with that :-) The problems I discovered was what happens if you pass an string containing markup to a filter etc. In my test cases I often screwed up things because a filter concatenated the markup object with another string with the result of having a plain string with the markup in. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Escaping in templates...
Hoi, -sys.maxint for autoescaping. I added support for that into jinja quite a while ago and it was pain in the ass. It makes things more complicated (speaking for return values and arguments of filters) and it blows up the implementation. Not worth the work. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #3527 - better debug traceback with code executing...
Hoi, -1 for the database interaction. The debugger should also work if the database is fucked up. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Dictionary lookup with variable in template
Hoi, What about using the Jinja approach? {{ foo.bar.baz }} is the same as {{ foo['bar']['baz'] }} So it's possible to use variables using this (where bar is a variable): {{ foo[bar].baz }} Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #3527 - better debug traceback with code executing...
Hoi, Alright. Patch here: http://pocoo.org/~mitsuhiko/django_debugger.patch Screenshot here: http://pocoo.org/~mitsuhiko/djangodebug.png (Note, I edited those files in the same checkout I did the patches for the __loader__ hook. So the patch is a bit bigger. Sorry for that) Enable by adding ENABLE_DEBUGGER = True into the settings.py file Also note: I had to modify the CommonMiddleware. That's not the best way to hook the traceback AJAX call in, maybe there are better ways. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: #3527 - better debug traceback with code executing...
Hi, On Apr 9, 1:48 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > Instead, it is triggered by whether or not DEBUG is True. Sometimes you > want to have DEBUG=True in production environments, whether for just a > little period of time -- to debug something -- or for longer. So I am > reluctant to put in something that might be a security hole if there's > any chance of it being run on a production site. The debugger must be additionally of course. Something like ENABLE_DEBUGGER = True/False in the settings. I would say the debugger is something that is missing in django. But it should be optional. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Status of SQLAlchemy branch
Hoi, On Mar 29, 9:07 am, "Mir Nazim" <[EMAIL PROTECTED]> wrote: > Now can anybody tell me about the status of Django's SQLAlchemy > branch. I could not find any place describing the status on > code.djangoproject.com. Looks like it's pretty dead. Very sad. SQLAlchemy is kicking-ass, especially with more complex queries. I would love to see some love in that branch. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
__loader__ support for debugging module (#3734)
Hoi, I filed this ticket (http://code.djangoproject.com/ticket/3734) some time ago and it's still marked as "Design decision needed" :). Now i added a patch and it works for me. I don't see any disadvantages in it so i hope someone can apply that patch. If not I would love to read comments about it. Short summary: this ticket adds support for __loader__ which is defined in the import hook pep and useful to retrieve sources from packages that are imported using import hooks (eggs for example) and also some template engines (currently afair only jinja) which use __loader__ to send debugging sources into the traceback formatter. Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Unbundling the ORM
The django ORM is the best out there. The only problem is that it's only working from inside django. There is a ticket for that: http://code.djangoproject.com/ticket/1321 and a discussion here: http://groups.google.com/group/django-developers/browse_thread/thread/5b590b5487e500c1/c0f0d30cec89b107?q=orm&rnum=5#c0f0d30cec89b107 Currently there are only two usable ORMs out there. SQLObject which isn't pep8 compatible and isn't so nice to work with as the django one and SQLAlchemy which is very complicated. For my current non django project i would like to use the django magic-removed ORM because it's great to use. The only problem is that using it require a complete django installation which isn't that good. So... Are there any plans to completly unbudle the ORM from django? Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Jinja Template Engine
I havn't compared with django but with some other template systems: [EMAIL PROTECTED]:~/Developement/area51/tpltest$ python tpltest.py Running Test 'jinja' TIME 0.00307821240425 Running Test 'kid' TIME 0.00703014059067 Running Test 'simpletal' TIME 0.0100415547848 Running Test 'cheetah' TIME 0.00251657156944 These tests where taken by enabling caching on all systems except simpletal* (kid/cheetah letting compile their .pyc files, jinja by creating a nodelist dump) I could compare it with django too, but i don't have some results by now. *simpletal doesn't provide an caching system? Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
{% recurse %} Tag
Hiho :-) Yesterday I tried to implement a sitemap with jinja and i encountered a missing tag :-) The {% recurse %} tag. So whats this all about? A small example is here: http://wsgiarea.pocoo.org/trac/browser/jinja/trunk/unittests/tec/recurse.py The generated output is here: http://wsgiarea.pocoo.org/trac/browser/jinja/trunk/unittests/tec/recursive_output.html?format=raw If someone finds that useful I could reimplement it in django template too, because I wouldl like to give something back to django :-) Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Jinja Template Engine
hugo wrote: > >Jinja isn't compatible with django templates. It uses a parser for the > >tag arguments > > A parser for tag arguments is available in Django, too - not all > template tags use only regexps. :-) You're right. Think i've overlooked it :-) Regards, Armin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Jinja Template Engine
Adrian Holovaty wrote: > On 2/22/06, Ivan Fedorov <[EMAIL PROTECTED]> wrote: > > What is Jinja? > > > > Jinja is a text-based template language similar to cheetah and Smarty. > > The syntax and parts of the code where taken from the django template. > > > > http://wsgiarea.pocoo.org/jinja/ > > Hey Ivan, > > What are the differences between Jinja and Django's template system? > At a glance, I see little things such as different available default > template filters and the fact that you have to use stuff like > FileSystemLoader()? Jinja isn't compatible with django templates. It uses a parser for the tag arguments and features some other syntax elements like {% marker %} / {% range %}... For my colubrid projects I wanted to have a django spin off with a improved argument handling. For example it's possible to do this: {% for variable in list %} {{ variable.text | replace "demo", "other" | escapexml }} {% print variable.value | escapexml %} {% endfor %} Heiko Wundram worked hard on a easy to extend argument parser which is explained here: http://wsgiarea.pocoo.org/jinja/docs/tag-dev.html > Rather than creating a separate project, why not just integrate your > changes into Django templates proper? We're going to make them > downloadable as a standalone package for Django's next version. Because it's a complete rewrite. I started it for improving my python skills and because I was sick of kid and cheeath ^^ I don't think that it can replace / should replace the current template sytem. Regards, Armin appendix: because of my bad english i've copied and modified some parts of the django template documentation. Sorry for that, I will rewrite those parts in the next days. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: GvR prefers Django Template over Cheetah And Modularised Django
I've created a ticket for this task: http://code.djangoproject.com/ticket/1321
Re: speeding Django up
Does django provide a system for caching the nodelist? In Jinja I've added a loader which cpickles the nodelist using protocol 2 which is very fast because you don't have to parse it again. Additonally you can store the templates in an global dict like in the solution proposed in that thread: http://groups.google.com/group/django-users/browse_thread/thread/40ecfffa8aa85a21/759da88e6d075ad7