from:"Ned Batchelder"

Re: Proposal to format Django using black

2019-04-15 Thread Ned Batchelder

On 4/15/19 7:13 AM, René Fleschenberg wrote:

HMS = "%H:%M:%S" # 14:30:59
HMSF = ".."
HM = ".."
TIME_INPUT_FORMATS = [HMS, HMSF, HM]

Just my two cents: This particular rule is the main reason why I
personally have not adopted black yet. I really find the
one-item-per-line style *much* more readable in many cases.

In natural-language writing, there are occasions where you use "foo,
bar, baz" and other occasions where you use

- foo
- bar
- baz

IMO, both variants have their place in Python code, too. I'd like to see
a black variant which allows to leave this decision to a human.

Try the latest black. A multi-line lists with comments on each line is
not converted to a one-line list. The lined-up indentation of the
comments is lost :(, but it's still multi-line.

This is one of my concerns: Black is still beta, and is still adjusting
the way it formats in small ways. So a boil-the-ocean formatting of the
code may not be the last time the code will change to suit black.

--Ned.

--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/19e8938e-47be-d1f2-6349-bf4a9eb506de%40nedbatchelder.com.
For more options, visit https://groups.google.com/d/optout.

Re: Module Index

2016-01-26 Thread Ned Batchelder


On 1/25/16 10:05 PM, Doug Epling wrote:
shouldn't this  be here: 
https://docs.djangoproject.com/en/1.9/py-modindex/#t


The module index is built from directives like ".. module: 
django.template.loader" in the various .txt files, like 
docs/topics/templates.txt.  Adding more directives will add entries to 
the module index.


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/56A762B9.1070901%40nedbatchelder.com.
For more options, visit https://groups.google.com/d/optout.

Re: structural & functional review of django documentation

2016-01-02 Thread Ned Batchelder

Doug, I'm having a hard time understanding you. You are pointing at data 
and coming to completely different conclusions than I do. Below you say, 
"The Django cadre must regularly ask about the state of public sentiment 
and satisfaction, because it is reckless to do otherwise."  This is 
after you examined the 3000 (!) responses to a survey of the Django 
community. Is there some other way the Django team should ask about 
public sentiment?


You also say, "If the consensus is to deny a need, the documentation 
will continue to be an afterthought," after you yourself noticed that 
the word "documentation" was a prominent answer to, "What's your 
favorite thing about Django?"  Seems like the public has been surveyed, 
and they like the docs.


You've started a discussion here, and in response, people have come up 
with concrete improvements to the docs, and are discussing other ideas.  
But you still seem to be approaching this as if 1) the docs are widely 
regarded as bad, and 2) nothing will be done about it.


Am I missing something?

--Ned.


On 1/1/16 5:35 PM, Doug Epling wrote:


I know some might have hoped I would just go away. But generally 
speaking when I say I will do something I follow through. At the very 
least I can work on the Glossary.



I looked at the poll of developers from last May. I loaded the results 
in an R data.frame, but I did not find any relationships within that 
data at all. I wonder what conclusions the core team was able to draw 
from that other than, like, 77 percent of the responses came within 48 
hours of its release. This fact must mean something,



Below is a wordcloud representation of the frequency of most used 
words under the "What's your favorite thing about Django?" item.



This doesn't really mean a lot, but it is kind of neat to look at.


You can notice the prominance of 'documentation', and 'orm'.


Again, these probably don't mean a whole lot, although developer folks 
sure exhibited an eagerness to express themselves. And you only need 
to skim over those results to see that a lot of Django regulars, the 
developers, really like the documentation. It would be interesting to 
hear why or how these folks use documentation that causes them such 
affinity for the docs.



Without those why-or-how answers to user interface questions, users 
defined as extremely active members of the developer community, it is 
hard to balance with the criticism that pops up here-and-there, 
including my own.



This discussion begs to transpire among members of the core team 
because nothing can change unless they see fit. If the consensus is to 
deny a need, the documentation will continue to be an afterthought.



The Django core developers are not the only public involved. Some 
might say they are in service to the public at large. The Django cadre 
must regularly ask about the state of public sentiment and 
satisfaction, because it is reckless to do otherwise.


Rplot_pos.png


On Sunday, December 27, 2015 at 5:42:49 PM UTC-5, Tim Graham wrote:

Adding a survey link is not difficult. We conducted a community
survey [1] earlier this year with one question related to
documentation, "What parts of the Django documentation do you find
most useful?" What questions to ask and how to turn the answers
into actionable items is the part I'm not sure about and maybe you
could advise on.

In my view, Django's docs haven't strayed from the "topics",
"reference", and "how-to" division that we've had since 1.0 or so.
Are you aware of this grouping? Maybe a "how the docs are
organized" section on the index page would help communicate that
and make it more intuitive where to look for something?

I'll admit I'm skeptical of a wholesale reorganization to this
structure for several reasons:
1. It'll be confusing for existing users who are familiar with the
current section.
2. It'll make it more difficult or impossible to backport
documentation enhancements to the stable version of the docs
(assuming we don't also reorganize them with same structure)
3. It'll create an opportunity for broken links (obviously we
could mitigate this to some extent by adding redirects to the new
locations).

It seems to me you were pretty close to finding what you were
looking for at https://docs.djangoproject.com/en/1.9/ref/forms/
 (first bullet,
I think), but I didn't understand what you meant by the page being
"the Joy of Cooking with Django."

[1]
https://www.djangoproject.com/weblog/2015/may/07/community-survey/


On Sunday, December 27, 2015 at 2:47:35 PM UTC-5, Doug Epling wrote:

Again, I am sorry if my comments have ruffled anyone's
feathers.  I am not going to argue.  My sole intent is a
positive one.  And, indeed, I am humbled by the

Re: Python 3.5 Support in Django 1.8.x?

2015-10-27 Thread Ned Batchelder

BTW, there's a move afoot to reconsider removing inspect.getargspec:
http://bugs.python.org/issue20438 and http://bugs.python.org/issue25486

--Ned.

On 10/24/15 10:30 AM, Tim Graham wrote:
Here's the PR to remove 50K lines of deprecation warnings when running
the Django 1.8 tests on Python 3.5 as well as to fix the two test
failures caused by those warnings:
https://github.com/django/django/pull/5472

In case you look and wonder about the build failures, Jenkins broke
today after a new release of sqlparse. I submitted a fix:
https://github.com/andialbrecht/sqlparse/pull/203

Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/f13c34f1-d059-4017-8626-41d6d5c3a760%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/563014A1.7000309%40nedbatchelder.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding more repr() methods

2015-08-30 Thread Ned Batchelder

The eval criterion rarely seems like a useful rule of thumb to me. Does
anyone actually use this to make objects? I think the useful rule is, a
repr should be an unambiguous representation of the object that is
useful in debugging. Certainly the "eval" rule is sufficient for the
"unambiguous" rule, but it isn't necessary. I would rather have
something readable than something evalable.

--Ned.

On 8/29/15 1:05 PM, Tim Graham wrote:
On the pull request, Fábio has raised the argument, "the |__repr__|
method should be unambiguous and |eval(repr(obj)) == obj|".

Is this something we should strive for?

On Friday, August 28, 2015 at 9:18:04 PM UTC-4, Josh Smeaton wrote:

Right, I agree with you in the case then. But I think the general
idea of more/improved repr methods is a good one.

On Friday, 28 August 2015 23:02:25 UTC+10, Tim Graham wrote:

The repr for MessageMiddleware includes settings.MESSAGE_STORAGE.

On Thursday, August 27, 2015 at 8:59:37 PM UTC-4, Josh Smeaton
wrote:

Generally, I'm a fan of adding repr methods. I don't think
we should run around finding all the places they might be
useful and adding them, but if someone is to put forward
patches where they've noticed a benefit, we should
encourage that.

I'm not sure I fully understand your concerns though. It
seems to me that the repr methods in that PR only include
things relating to the class being observed. Did I miss
something?

Cheers

On Friday, 28 August 2015 01:16:02 UTC+10, Tim Graham wrote:

I'd like to ask for other opinions on this pull
request which adds more __repr__() methods:
https://github.com/django/django/pull/5059

Some concerns:

* Adding things that aren't part of the class (e.g.
settings) seems questionable to me.

* There is a chance to leak sensitive data in
|__repr__| as done in #22990
(doesn't
seems to be a concern with the changes here, but is
something to be mindful of).

Thanks!

Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/3f26bb3a-5c3d-4057-a788-92cde6768afb%40googlegroups.com
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/55E30AB0.2050305%40nedbatchelder.com.
For more options, visit https://groups.google.com/d/optout.

Re: should we close in-memory file-like objects (StringIO, BytesIO, etc.)?

2015-07-04 Thread Ned Batchelder

On 7/3/15 1:55 PM, Aymeric Augustin wrote:
2015-07-03 17:10 GMT+02:00 Berker Peksağ >:

I agree with you on the StringIO and BytesIO cases, but I agree with
Andriy on the other usages (e.g. ZipFile, GzipFile). Many contributors
follow the existing practices to write new tests, so it would be nice
to use best practices if they don't produce too much noise.

StringIO and BytesIO are different from files and sockets. The former
only use
RAM while the latter use file descriptors, which are in much scarcer
supply

(perhaps 1024 per process).

Leaking file descriptors can cause the process to crash because it
exceed the

systems limit, so it's really bad.

Leaking StringIO or BytesIO doesn't change anything. They'll be closed
when

the garbage collector releases them.

That's why I don't believe the analogy holds.

Many of the StringIO and BytesIO changes in the pull requests won't
change anything. The with-statement extends to the end of the test
method, which is when the object would be reclaimed anyway:

@@ -31,10 +32,10 @@ def write(self, data):
self.bytes_sent += len(data)

# XXX check Content-Length and truncate if too many bytes written?

-data = BytesIO(data)
-for chunk in iter(lambda: data.read(MAX_SOCKET_CHUNK_SIZE), b''):
-self._write(chunk)
-self._flush()
+with contextlib.closing(BytesIO(data)) as data:
+for chunk in iter(lambda: data.read(MAX_SOCKET_CHUNK_SIZE),
b''):
+self._write(chunk)
+self._flush()

def error_output(self, environ, start_response):

super(ServerHandler, self).error_output(environ, start_response)

I don't think it makes sense to blindly use contextlib.closing on
anything that acts like a file, even when we know it is a harmless
StringIO. Better to have developers think about what is happening, and
use the appropriate construct.

When I see " contextlib.closing(BytesIO(data))", I think, this person is
following a pattern, and doesn't understand.

--Ned.

--
Aymeric.
--
You received this message because you are subscribed to the Google
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to django-developers+unsubscr...@googlegroups.com
.
To post to this group, send email to
django-developers@googlegroups.com
.

Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CANE-7mXog8YE8LMSxwokaMddjDjcK949wUNzEd_NnLtcknWE_A%40mail.gmail.com
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/5597D718.9090701%40nedbatchelder.com.
For more options, visit https://groups.google.com/d/optout.

Django template coverage experiment

2015-01-15 Thread Ned Batchelder


Hi,

I've added support to coverage.py for plugins to support non-Python 
files, and I've written a plugin to measure Django templates.  I'm 
looking for a collaborator to decide how to complete the work.  I'm 
asking on the dev list because ideally, the plugin would be part of 
Django itself, although it doesn't have to be.  There are also 
possibilities to change the template engine slightly to make coverage 
measurement of templates better.


If you'd like to try it, the plugin itself is pip installable: pip 
install django_coverage_plugin . To run it, add these settings to your 
.coveragerc:


   [run]
   timid = True   # won't be needed eventually
   plugins =
django_coverage_plugin

Then run your tests under coverage as you normally would.  It requires 
coverage.py>=4.0a2, so it may not work with other coverage-related tools 
if you have them, such as coveralls.io.  You will see your templates 
listed in your coverage report alongside your Python modules (they have 
a .html extension but no directory, that's still to be fixed).


The technique used to measure the coverage is the same that Dmitry 
Trofimov used in dtcov, but integrated into coverage.py, and made more 
performant.  I'd love to see how well it works in a real production 
project.  If you want to help me with, feel free to reply offlist if 
it's more appropriate.


Thanks,

--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/54B85A9E.8050500%40nedbatchelder.com.
For more options, visit https://groups.google.com/d/optout.

Re: [1.7-RC] Using coverage with migrated app slow down test-suite execution by 6 times.

2014-07-14 Thread Ned Batchelder


On 7/10/14 6:32 AM, Stan wrote:
I tried to found a revision in the /stable/1.7.x/ branch showing a 
regression in the time needed to run the test suite with Coverage.
Unfortunately I was unable to find such commit because with checkouts 
from the 12 of june and beyond, my test starts to break.


Conclusions: at this point, there is no clue that /Migration/ is 
responsible or not because the regression involves 3 parameters 
between /good /and /bad/:


 1. lot of changes in the Django codebase*;*
 2. the use of a /migrated/ app in the test project.
 3. changes in the test project codebase.

Some logs, for the record:

Using the parametrage migration file generated with 1.7-RC
==

Try to find the commit introducing a regression in the time needed to 
execute the test suite

inside coverage. A /Good/ revision needs t < ~0.125s.

1. checkout [commit] (Django 1.7.x branch) && python setup.py build && 
python setup.py install
2. coverage run --source='.' manage.py test 
fab4.fabrication.tests.tests.ParutionTest


Test pass, no difference in time needed
'''

30d8b95139a1fa3070f3b112115e624ffcf8e555 (19 june): 0.325s
f16554f440a9640dd619ac01278565e4e3e1e2c6 (17 june): 0.412s
6e248d8f8c95da9cbd654d43df76c20e6d0ec352 (16 june): 0.327s
a5fed757aaa1eebe4f22f8e2a6c47296e850712f (16 june): 0.319s
ec2f0417364050aed3e8c2aa1948915d08c39514 (14 june): 0.308s
d3bf537324c1164dee45070f90d48661b27a781f (12 june): 0.310s
83e96ee99b6cd8ce1863dcca24dfbf4a8d6b5429 (12 june): 0.332s
961c9d6c6b1975fba8eb1492288e0fc49652c43d (12 june): 0.351s

Exception raised when running the test
''

84714dfed726212122ac9a143ddbe6719dac5884 (12 june)
c6725d69a2b7f00653886f0aeae301b7ce0baf69 (11 june)
1ff11304dcebbabdede8ef3d659ca0e54055e2fd (11 june)
342b25449d800ce29ae56ff8285869cbc3133a75 (5 june)
5f135e6a0b77e76649aac48434740cc703e843ff (28 may)
157575c7c8ad4e55b68077c2d10b41d596038e2e (19 may)
9f5adf8db37280e8ddc093a36c9ad1c20142a5ca (Beta 14 may)

ValueError: Lookup failed for model referenced by field
fabrication.Dossier.presentation_annonceur: 
parametrage.PresentationAnnonceur



Regenerate the migrations file for parametrage
==

The same here but with a fresh migration file at each test iteration.

1. checkout [commit] (Django 1.7.x branch) && python setup.py build && 
python setup.py install
2. rm -rf ./fab4/parametrage/migrations && python manage.py 
makemigrations parametrage
3. coverage run --source='.' manage.py test 
fab4.fabrication.tests.tests.ParutionTest


Test pass, no difference in time needed
'''

534d9f1e822ef49550c195a4e9493970781c4591 (16 june - Added database 
migration for contrib.sites.): 0.345s
3ef87f664bb8b2a46546de755bfc4e158c90032f (15 june - Persist 
non-schema-relevant Meta changes in migrations): 0.320s
c903543127e642e619213cdefda9cea15df09c16 (15 june - Fixed #22563: 
Added migration to admin, fixed a few more load…): 0.316s


Fails on makemigrations parametrage
'''

ec2f0417364050aed3e8c2aa1948915d08c39514 (14 june)
- Error on (2) makemigrations parametrage:
KeyError: u"Dependency references nonexistent parent node 
(u'contenttypes', u'__latest__')"

- Error on (3) with the same exception.

7bd2ad1dd9fc449ed1b4832fab5c975d7c8aa895 (11 june)
- Error on (2) makemigrations parametrage:
- Error on (3): Cannot run test on unmigrated app.


Dig in 1.7.x branch for a revision that can actually run the test 
without migrating parametrage

===

The problem here is that my django-1.7 branch of my project is broken 
with old Django revisions.


1. checkout [commit] (Django 1.7.x branch) && python setup.py build && 
python setup.py install
2. coverage run --source='.' manage.py test 
fab4.fabrication.tests.tests.ParutionTest



25f4e71ed30befbd6af02a7d35807d0d3170e57d (8 june)
2dd52d043344c9e9acf65149d8c72b32b40f4643 (5 may)
35c2a14a49ac3cb25dcff818b280bf0b4c290287 (30 april)
edca57817fa5c366483194020967cd9d4ef1318c (28 april, beta3)
1bcc8eb0f64831b24c2769e033a10e530c8ed140 (10 april)
62ca2354abadf259f4186a0b43f583ed1bd1 (20 march)
c564277937a1f0b78b2c2b701e6381319d9685c2 (10 march)
5d568bcfa66916e3de61e0090c724c899debd981 (5 march)
f732d55dfcd06faf16a5d45798ae2f53d7784d66 (1 march)
476abc7ea72fc7cbfed8e5a7b50b5d82e5d32105
- Cannot run the test (parametrage is not migrated here) :
  KeyError: u"Dependency references nonexistent parent node 
(u'contenttypes', u'__latest__')"



2ebccebf0609229317c2f5b9a76664dc216f27eb (15 feb)
ImportError: Could not import settings 'fab4.settings' (Is it on 
sys.path? Is there an import error

in the settings file?): cannot import name constants


fb1e3435a4d7e0265f19a1a9f130c9485fb8dfe9 (10 feb)
  File

Re: Reconsider 20383: limited contexts for makemessages

2013-07-05 Thread Ned Batchelder


On 7/4/2013 5:40 AM, Benjamin Wohlwend wrote:

Hi,

I recently hit a problem where I have to provide translations for a 
reusable app. In German (and most other languages, except English, it 
seems), there are different second-person pronouns used in different 
situations, see [1]. As an example, the sentence "Do you want to log 
in" in its two German translations: "Wollen Sie sich anmelden?" 
(formal, polite), and "Willst Du dich anmelden?" (casual). Because my 
reusable app is used in all kinds of projects, ranging from a social 
network to a banking website. Using "Sie" on the social network is 
just as out-of-place as "Du" on the banking website.


What I did so far is something like this:

  {% if is_formal %}
{% trans "Do you want to log in?" context 'formal' %}
  {% else %}
{% trans "Do you want to log in?" context 'casual' %}
  {% endif %}



Perhaps I'm naive, but would it work to treat these as two different 
languages?  We already deal with Portuguese (Brazil) and Portuguese 
(Portugal), would it be possible for you to have German (Formal) and 
German (Informal)?  Of course, part of that label would be provided by 
the user (German) and part by the app (Formal), but that composition and 
selection of language could be done in one place in the app, rather than 
everywhere strings appear.


--Ned.

This gets very tiresome if you have more than a few translation 
strings. It's not DRY, and it's error prone. As it happens, ticket 
#20383 would provide the perfect solution for this, even if the use 
case therein is different. I guess Claude was right in his assertion 
that the white labeling use case might be to limited to warrant the 
additional complexity. The T-V distinction problem, on the other hand, 
is present in (AFAIK) all Latin languages, German, and many other 
languages. Is this enough reason to reopen that ticket?


Regards,
Benjamin

[1]: https://en.wikipedia.org/wiki/T%E2%80%93V_distinction
[2]: https://code.djangoproject.com/ticket/20383
--
You received this message because you are subscribed to the Google 
Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com.

To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.




--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANNOUNCE: Django 1.6 beta 1 released

2013-06-28 Thread Ned Batchelder



On 6/28/2013 9:48 AM, Jacob Kaplan-Moss wrote:

Hi folks --

I'm pleased to announce that we've just released Django 1.6 beta 1,
the second in our series of preview releases leading up to Django 1.6
(due in August).

More information can be found on our blog:

https://www.djangoproject.com/weblog/2013/jun/28/django-16-beta-1-released/

And in the release notes:

https://docs.djangoproject.com/en/dev/releases/1.6/



The 1.5 release notes said that Python 3 support was experimental, and 
that the next release would support Python 3 "without reservations."  
The 1.6 release notes don't mention Python 3 at all.  Shouldn't the full 
support be included in the release notes?


--Ned.


Thanks!

Jacob
--
You received this message because you are subscribed to the Google 
Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com.

To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.




--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Redesign of djangoproject.com?

2012-04-28 Thread Ned Batchelder

Will there be any specific discussion about what's wrong with the 
current site?  You two seem to agree something needs to be done, but 
haven't mentioned anything specific.


--Ned.

On 4/28/2012 4:13 AM, Dana Woodman wrote:
Great to know they're is some interest in it and agreement that it is 
in need :)


I'm very interested in the prospect of redesigning the site and would 
love to chat more about it. I feel like I owe the Django community 
something for all that it has given me (including the job I currently 
have!). I'd love to talk scope and other factors so I have a clear 
idea of what would be involved.


Is this a good forum for this type of discussion or should we get in 
touch elsewhere to talk? You can get in touch with me here: 
http://danawoodman.com/


--
Dana Woodman
d...@danawoodman.com 
http://www.danawoodman.com

On Saturday, April 28, 2012 at 1:05 AM, Russell Keith-Magee wrote:


Hi Dana,

I completely agree. I've been trying to get a redesign of 
djangoproject.com  going for quite some 
time under the auspices of the Django Foundation. As you can see from 
the lack of changes, you can see that I haven't been particularly 
successful :-(


The fundamental problem is that we have plenty of coding talent at 
our disposal, but not as much design talent. That's not to say that 
there aren't many talented designers in our community -- there are -- 
it's just that they're all very busy. We've approached several people 
in the Django design community asking them to help out, and some have 
even made done some initial work. However, redesign of a high-profile 
site like djangoproject.com  is a big job, 
and nobody has been able to spare the time to bring the job to 
completion.


So - at this point I'm open to any offers. I want to avoid design by 
committee -- ideally, I would like to pass this off to a single 
person (or a small group) and give them complete control over design 
process. I'm not completely sure how to organise who gets this role 
-- suggestions are welcome.


If you (or anyone else) is interested, drop me a line and I can give 
you the design brief we've been working with.


Yours,
Russ Magee %-)



On Saturday, 28 April 2012 at 3:22 PM, Dana Woodman wrote:

So now that Django is being moved to Git/Github (which is awesome!), 
maybe it would be a good time to think about a revamped home page 
for the project ala djangoproject.com (http://djangoproject.com)?


Obviously this is no small undertaking and would be potentially 
contentions as to what would be the proper path, but I feel (and I 
don't think I'm alone) that djangoproject.com 
(http://djangoproject.com) could use a bit of a facelift.


I have some idea of my own as to how this could be accomplished and 
I'm sure there are a ton of others out there with great ideas as 
well. Maybe we could open up some discussion on this idea?


Forgive me if this has been proposed before as I'm new to the group!

Cheers,
Dana

--
You received this message because you are subscribed to the Google 
Groups "Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/g8ngEnVG_EsJ.
To post to this group, send email to 
django-developers@googlegroups.com 
(mailto:django-developers@googlegroups.com).
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com 
(mailto:django-developers+unsubscr...@googlegroups.com).
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.




--
You received this message because you are subscribed to the Google 
Groups "Django developers" group.
To post to this group, send email to 
django-developers@googlegroups.com 
.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com 
.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.


--
You received this message because you are subscribed to the Google 
Groups "Django developers" group.

To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting multiline tags

2012-02-27 Thread Ned Batchelder


On 2/26/2012 12:12 AM, Yo-Yo Ma wrote:

After Ned's message, I'm -0, because while I'm not fond of multi-line
tags, I cannot offer a good alternative when it comes to multi-line
"with" tags.

On Feb 25, 6:48 pm, Ned Batchelder<n...@nedbatchelder.com>  wrote:

On 2/24/2012 11:55 PM, Yo-Yo Ma wrote:>  I'm -1 on this for s specific reason; 
If you need multiple lines for a

tag, you're doing it wrong.

import this

This would be far more helpful feedback if you would take the examples
of too-long tags presented in this thread, and show the "right" way to
do it.

--Ned.
While I'm glad to see people being open to changing their minds, it 
worries me that none of the "one-line tag" aesthetes have spoken up to 
explain how they would deal with the unwieldy constructions that started 
this thread.  In fact, so far two people have changed their minds when 
actually grappling to come up with an answer.


What is the right way to do this:

{% trans with
 varname=myobject.proprety1
 someothervar=myobject.some.other_property
 yetanothervar=myotherobject.with_a_painfully_long_method_name
 "Even with line-wrap, it's a pain to read on a single line."
%}

or

{% blocktrans with originator=entry.originator|get_really_full_name:"mailto" 
link=entry.get_metadata.link task_id=entry.task.id 
process_name=entry.task.process_class|contenttype_name %}

Powers-that-be declaring, "X is ugly and won't happen" is inevitable, 
but we should be able to extend the answer with "and Y is the way to do 
it better?"


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting multiline tags

2012-02-25 Thread Ned Batchelder


On 2/24/2012 11:55 PM, Yo-Yo Ma wrote:

I'm -1 on this for s specific reason; If you need multiple lines for a
tag, you're doing it wrong.


import this
This would be far more helpful feedback if you would take the examples 
of too-long tags presented in this thread, and show the "right" way to 
do it.


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Improving test data experience

2011-11-30 Thread Ned Batchelder




On 11/30/2011 5:22 PM, Jeremy Dunck wrote:

Hi all,
   I've got a codebase with a fair bit of fixtures and have been
experiencing some pain around it.  One of the pains is migrating the
format of fixtures when a schema change occurs.

   I posted a thread to south-users to discuss the idea of adding a
feature to aid applying migrations to fixture data:
   
http://groups.google.com/group/south-users/browse_thread/thread/e358d13a55545082

   Evgeny responded that perhaps scripting generation of the fixture
was a good approach, and Carl responded that he thinks that generating
the needed test data in the TestCase itself is preferable.  Both
approaches avoid the immediate problem of wishing I could migrate
fixture data, because generating the data under their approaches
avoids having fixtures as an authoritative source of data.

   That surprised me a bit since I generally have viewed fixtures as
things to set up once.  Have others on the list tried these approaches
(or other approaches) and have any thoughts you'd like to share on it?

In any case, I think it would be good to do one or both of the following:

1) expand on the testing guide to present fixtures as one option
for test data and point out the options to script fixture generation
or avoiding fixtures in favor of TestCase-called generation,
discussing tradeoffs in the approaches
2) add a django-admin testshell command which would act like
testserver - loading fixtures into a test DB and dropping you into the
shell to munge it as needed; this would be paired with a south feature
for applying migration fixtures.

   Feedback, please?
FWIW, this is the process I've used in the past:  
http://stackoverflow.com/questions/4002322/migrating-django-fixtures   
There's got to be a better way.


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

2011-06-08 Thread Ned Batchelder


On 6/6/2011 10:19 PM, Ramiro Morales wrote:

On Sun, Jun 5, 2011 at 5:18 PM, Ned Batchelder<n...@nedbatchelder.com>  wrote:

When I try this on a PostgreSQL database, I have problems relating to
violated uniqueness constraints, sometimes from tests themselves, sometimes
from setUpClass, sometimes from tearDownClass.  In the latter two cases,
it's the sites table involved.  Is this something others have dealt with, or
am I on my own? :)

I tried adding a PostgreSQL "disable constraints" statement here:
https://github.com/jbalogh/test-utils/blob/master/test_utils/__init__.py#L109

cursor.execute('SET CONSTRAINT ALL DEFERRED')

It didn't help.

This might be related to ticket [1]#11665, a knownissue in the TestCase
handling of constraints with pg. Suggestion athere si to use
SET CONSTRAINTS ALL IMMEDIATE
before the rollback.

HTH

Thanks for the idea, but it did not help, which makes sense to me.  The 
ticket in question is complaining that TestCase doesn't throw enough 
constraint violation exceptions, and suggests setting SET CONSTRAINT ALL 
DEFERRED as a way to force problems to the surface.  I'm trying to do 
the opposite...


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

2011-06-05 Thread Ned Batchelder


On 5/17/2011 2:28 PM, Erik Rose wrote:

I would be very happy to test this against Oracle database to see is how
much patch improves speed since previously running tests against Oracle
has been a real pain specially all db recreate stuff took a long long
time.

Great! I'll post again to this thread when the patch is ready. Or, if you'd 
like to try it now, you can download https://github.com/jbalogh/test-utils and 
make your test classes subclass FastFixtureTestCase rather than TestCase.

When I try this on a PostgreSQL database, I have problems relating to 
violated uniqueness constraints, sometimes from tests themselves, 
sometimes from setUpClass, sometimes from tearDownClass.  In the latter 
two cases, it's the sites table involved.  Is this something others have 
dealt with, or am I on my own? :)


I tried adding a PostgreSQL "disable constraints" statement here: 
https://github.com/jbalogh/test-utils/blob/master/test_utils/__init__.py#L109


cursor.execute('SET CONSTRAINT ALL DEFERRED')

It didn't help.

Thanks,

--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-06-03 Thread Ned Batchelder


On 6/3/2011 12:26 PM, Jannis Leidel wrote:

On 03.06.2011, at 04:15, Ned Batchelder wrote:


On 5/29/2011 5:40 AM, Jannis Leidel wrote:

On 15.04.2011, at 15:25, Ned Batchelder wrote:

Hi Ned,


As you say, Jannis has suggested that a Babel-based solution isn't that much 
work.  But that work hasn't been done yet.  I don't know how much work it is.  
It's going to be a larger change to the code base than my patch is, at least it 
is if you properly consider that the Babel code is part of the change, even if 
it isn't included in the patch.  But we don't have a Babel patch to consider, 
only the suggestion that it won't be a big deal and it will work well.  That 
suggestion remains to be proven.

I have good and some bad news. Last week I tried to put my money where my mouth
is and dived into Babel to replace the xgettext calls in makemessage with the
appropriate calls to Babel's message extraction/compilation functions. I've
found it to be pretty easy to get started at first but stumbled over differences
in the way the message catalogue update process works.

Contrary to Django it doesn't easily allow to "look for new strings and update
PO file" but needs a separate step to update an original POT, which is then to 
be
manually merged with the existing PO file. Given the amount of glue code I had 
to
write to work around it, I'm not longer convinced that the adoption of Babel
would be a sensible to fix to the Javascript message extraction problems.


I'm sorry to hear Babel didn't work out, it would have been liberating to have 
a tool entirely within our grasp.

That said, I'd be happy to know whether there are any updates with regard to 
your
Javascript lexer that I need to consider before merging your patches.

There were two fixes to jslex.py, I've added an updated patch to ticket #7704.  
Let me know what I can do to help with the merge.  Thanks.

Great, thank you, I'll try to get this in during the next week's DjangoCon 
sprints.

Just out of curiousity though, is the update basically
https://bitbucket.org/ned/jslex/changeset/2270dbe90afe and
https://bitbucket.org/ned/jslex/changeset/4f4bc539f533?


Yes, that's exactly what it is.

--Ned.

Jannis



--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-06-02 Thread Ned Batchelder


On 5/29/2011 5:40 AM, Jannis Leidel wrote:

On 15.04.2011, at 15:25, Ned Batchelder wrote:

Hi Ned,


As you say, Jannis has suggested that a Babel-based solution isn't that much 
work.  But that work hasn't been done yet.  I don't know how much work it is.  
It's going to be a larger change to the code base than my patch is, at least it 
is if you properly consider that the Babel code is part of the change, even if 
it isn't included in the patch.  But we don't have a Babel patch to consider, 
only the suggestion that it won't be a big deal and it will work well.  That 
suggestion remains to be proven.

I have good and some bad news. Last week I tried to put my money where my mouth
is and dived into Babel to replace the xgettext calls in makemessage with the
appropriate calls to Babel's message extraction/compilation functions. I've
found it to be pretty easy to get started at first but stumbled over differences
in the way the message catalogue update process works.

Contrary to Django it doesn't easily allow to "look for new strings and update
PO file" but needs a separate step to update an original POT, which is then to 
be
manually merged with the existing PO file. Given the amount of glue code I had 
to
write to work around it, I'm not longer convinced that the adoption of Babel
would be a sensible to fix to the Javascript message extraction problems.

I'm sorry to hear Babel didn't work out, it would have been liberating 
to have a tool entirely within our grasp.

That said, I'd be happy to know whether there are any updates with regard to 
your
Javascript lexer that I need to consider before merging your patches.
There were two fixes to jslex.py, I've added an updated patch to ticket 
#7704.  Let me know what I can do to help with the merge.  Thanks.


--Ned.


Jannis



--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

2011-05-17 Thread Ned Batchelder


On 5/17/2011 11:31 AM, Jeremy Dunck wrote:

On Tue, May 17, 2011 at 10:24 AM, Ned Batchelder<n...@nedbatchelder.com>  wrote:

Maybe it wouldn't be so bad to punt on invalidation?  The cached databases
would only have to be rebuilt if the models changed or if the fixtures
changed, right?  We have a similar situation now with migrations: you have
to write one every time you change a model, and there's no automatic
mechanism that kicks in to tell you to write one, you just have to know:
"change a model, write a migration."  If that's working now, then what's
wrong with, "change a model or a fixture, re-run the test database cacher."

The difference is, migrations can be merged.  Database cache is local
state.  No?


Hmm, that's a good point.

--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

2011-05-17 Thread Ned Batchelder


On 5/17/2011 10:48 AM, Jeremy Dunck wrote:

On Tue, May 17, 2011 at 9:16 AM, Jonas H.  wrote:


Invalidation is what I'm unsure about too -- multiple ideas came to my mind,
all involving some sort of Great Hash(tm):

Even within a single test command run, the same DB setup and same
fixture loads are done many times (for a sizable suite).  Invalidating
too often is better than invalidating too little.


1) Use file modification timestamps for all model and test related files.
Advantages: simple, works.
Disadvantages: Triggers cache invalidation for changes not related to models
or tests

I think this is a pretty big win, even though it's not theoretically optimal.

Maybe it wouldn't be so bad to punt on invalidation?  The cached 
databases would only have to be rebuilt if the models changed or if the 
fixtures changed, right?  We have a similar situation now with 
migrations: you have to write one every time you change a model, and 
there's no automatic mechanism that kicks in to tell you to write one, 
you just have to know: "change a model, write a migration."  If that's 
working now, then what's wrong with, "change a model or a fixture, 
re-run the test database cacher."


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

2011-05-17 Thread Ned Batchelder


On 5/16/2011 11:18 PM, Erik Rose wrote:

How about caching the test databases? The database state could be cached after 
model setup (which takes some time if you've got lots of them) + initial data 
fixture setup, and after the setup for each test case (fixtures + setUp() 
method).

So, in the best case, no database setup is required at all to run tests -- 
which encourages test driven development :-)

So that would be 11 separate DBs for our tests, and you'd just switch between 
them? Interesting idea. Or are you proposing caching the results of queries for 
each test class, essentially mocking out the DB?

I'd been thinking recently about this as well: when you consider all the 
test runs, they're very repetitive.  Every time the tests are run, they 
go through the same set of steps: a) create database, b) install 
fixtures, c) run tests.  Steps a, b, and c take too long.  Step c is 
what we're really interested in, and almost always, steps a and b have 
the same outcome as the last time we ran them.  We all know what to do 
if an operation takes too long and usually is the same as last time: 
cache its outcome.  The outcome in this case is the state of the 
database.  Caching it could be as simple as making a copy of the 
database after the fixtures are installed, then using that copy to run 
tests.


The complications are: 1) in any interesting test suite, there isn't a 
single outcome of a+b, because different tests will have different 
fixtures and perhaps even different models, so a number of copies will 
have to be captured.  2) As with any caching scheme, invalidation is 
important and tricky.  In the normal course of development, how will 
these cached copies of the database be invalidated and recreated?  
Perhaps this isn't so bad, it's roughly analogous to writing migrations, 
which we know how to deal with.


I don't have any code to do this, but I envision a set of test 
databases, with a modified test runner than knows how to cycle among 
them by manipulating settings.DATABASES to use the proper one for each 
test class.  I'd be glad to help build such a thing, and may be working 
toward it myself.


--Ned.


Perhaps some numbers would illuminate: I clock the total setup and teardown 
time for support.mozilla.com's 1064 tests at 2.59 seconds after my 
optimizations. So I'm pretty happy with that. :-) CPU use for the test run is 
76% according to the `time` commands, so there's a little more I/O to kill but 
not much.

Cheers,
Erik



--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-04-17 Thread Ned Batchelder

On 4/15/2011 12:40 PM, Russell Keith-Magee wrote:

On Fri, Apr 15, 2011 at 9:25 PM, Ned Batchelder<n...@nedbatchelder.com> wrote:

On 4/14/2011 11:40 PM, Russell Keith-Magee wrote:

Keep in mind that the proposal is not to include Babel, but to depend on it
as a prerequisite, which means we are stuck in the same situation we are
with gettext: it can change independently of Django, and new versions can
introduce new bad behavior. That's one of the reasons we have a bad problem
today: gettext changed from 0.17 to 0.18, and exacerbated the hack.

This is true. However, what you're proposing is, IMHO, a slightly
worse situation.

Babel is a self contained tool. Assuming it work as advertised (and
I'll grant that is a big and important assumption), it is a self
contained body of code. It is a dependency, but as long as it
continues to work as advertised, we're fine. We're only dealing with
it's advertised interface, and working with that interface in the way
it was intended to be worked with.

On the other hand, gettext is also a dependency, and gettext can also
change between releases -- but we're not using it as intended. We're
bending the Perl parser (or, in your case, the C parser) in strange
and unusual ways to do something it wasn't originally intended to do.
Something completely innocuous can change in gettext, and the
follow-on effect to us can be huge because we've built our castle on
an unstable foundation.

I'm not much concerned that the C parsing in gettext will change
significantly, but I take your point, it certainly could change behind
our backs and we could be broken again.

The maintenance issue is the critical part here. My hesitation isn't
just to do with the suitability of your code *right now*. It's to do
with the fact that once we adopt the code into trunk, we are to
agreeing to maintain it. Bits don't rot, but gettext has already
demonstrated that it changes between versions, so it's reasonable to
assume that when gettext 0.19 is released (whenever that happens),
we'll need to make changes to our Javascript parser. By taking on the
lexer, we're absorbing into Django a whole bunch of project
responsibility that frankly, I'd rather we didn't have.

Babel
has the advantage that it is pure Python, so it is both more installable
than gettext, and is more readable for us. It also has the advantage that
it isn't based on a hack, but that doesn't mean it performs flawlessly.

If the proposed patch was leaning on a well established lexer, or was
a simple configuration change (i.e., "treat it as C, not Perl") that
could be quickly demonstrated to fix a bunch of problems without
introducing any obvious new problems, I'd be all in favor of it as a
temporary solution. But that's not what is on the table -- it's a
complex body of code that we're proposing to evaluate, introduce,
maintain, and potentially deprecate very quickly.

While the patch I've submitted is certainly larger than a configuration
change, and is not a well-established lexer, I have "quickly demonstrated
that it fixes a bunch of problems without introducing any obvious new
problems", or at least, no one has come forward with a new problem. I've
paid a bounty on Stack Overflow for people to find problems in the lexer
itself, which they have done, and those problems have been fixed.

That sort of thing evidence certainly works in your favor -- I wasn't
aware that this sort of testing had taken place.

I'm happy to be proven wrong on any of the points mentioned here --
for example, if someone can provide some mechanism that independently
demonstrates the robustness of the Javascript lexer, or evidence that
the risk of regressions is low. However, absent of such evidence, I'm
inclined to side with Jannis and concentrate our efforts on Babel --
especially if, as Jannis suggests, a Babel-based solution isn't that
much work and could be knocked off easily with a short concentrated
effort.

I'd be glad to undertake the effort to demonstrate the robustness of the
Javascript lexer, if someone can tell me what that test would look like.
I've done the work to read the ECMAScript spec, I can certainly do the work
to write more tests. I've run some significant code through the lexer
(jQuery, for example), and it didn't result in any 'other' tokens, and was
properly synchronized at the end, though I can't say I examined every token
in the stream. If anyone has an idea how to more thoroughly test a lexer,
I'm all ears.

Again -- this is good evidence, and something that hasn't (AFAICT)
been stated previously in this forum.

Sorry, I also wrote about this journey on my blog
(http://nedbatchelder.com/blog/201104/a_javascript_lexer_in_python_and_the_saga_behind_it.html),
and had lost track of which details went where.

But keep in mind: that work will also have to be done for Babel. I'm more
than happy to contribute my 216 lines of tests to their 89 lines, or to lend
my new-found knowledge about

Re: Fixing makemessages for Javascript

2011-04-15 Thread Ned Batchelder


On 4/14/2011 11:40 PM, Russell Keith-Magee wrote:

On Fri, Apr 15, 2011 at 12:30 AM, Jannis Leidel  wrote:

On 14.04.2011, at 17:27, Jacob Kaplan-Moss wrote:


I think I agree with Ned here: I can't see the downside to fixing it
on the release branch. "It violates our policy" doesn't count IMO:
it's *our* policy, and we get to break it if there's a good reason.
Making translation in JavaScript work is a good reason as I see it.

FTR I agree we have an issue here, I just disagree with the proposed fix. If 
you think we can adopt Babel in the release branch, let's do it.


Jannis: can you speak a bit more to why you're against fixing this for
the 1.3 series? Is there a technical downside I'm missing?

Again, I'm not convinced that Ned's patch solves the actual problem: Django abusing the 
xgettext CLI tool to parse JavaScript files with its Perl lexer. He proposes to convert 
the content of JavaScript files with a custom made lexer to a format that xgettext can 
understand as "C" instead. This -- while being a nifty piece of code -- seems 
like a hack to me. Which is why I believe that such a code only adds more maintanance 
burden on us for the already fragile i18n system and should be replaced with a proper 
JavaScript parser.

So I've proposed to adopt Babel, which is a proven, widely used system that 
implements many of the weird hacks we have in Django i18n cleanly in Python. As 
a bonus it would allow us to get rid of a binary dependency that was difficult 
to install on all platforms in the past anyway.

In other words, technically we're speaking of a bikeshed that has some holes in 
the roof and Ned is trying to fix them with new paint. IMHO, it needs to be 
reconstructed instead. Whether we do this in the release branch or in trunk I 
don't care.

I'm a little uncertain where to throw my vote here.

I can appreciate Ned and Jacob's "pragmatic" approach -- this problem
exists, so an interim partial solution is better than nothing if it is
easy to apply.

However -- I can also see Jannis' point: Babel is the real solution
here, and any effort spent on maintaining the existing lexer is effort
that could be spent in fixing the problem properly with Babel.

My concern in accepting the "pragmatic" approach is whether it
actually *is* pragmatic. Completely independent of whether there is a
Babel solution coming soon, I don't have any feeling as to whether
Ned's proposed partial solution will introduce more problems than it
solves.

My gut feel tells me the Javascript is more like C than Perl, but
that's really nothing more than a gut feel, and the proposed patch
does much more than just say "treat it like C" -- it also involves
introducing a JavaScript lexer. We're proposing putting this into the
1.3 branch -- a stable branch -- without any significant testing.
We're essentially saying as a project that the JS lexer is stable
tested code, known to be better than the existing solution in all
conditions, with no significant regressions.

No offense intended to Ned, but I simply don't see sufficient evidence
to be confident that this is the case. 216 lines of tests doesn't
strike me as anything close to a broad enough test suite to validate
that the Javascript lexer will work under all conditions. Yes, the
Perl-based lexer is broken, but it's broken in known ways; we don't
have any experience to know where the flaws lie in the C-based lexer,
and once it's in Django's repo, we're committing to maintaining it and
all it's flaws and foibles.

I'm don't know why we are assuming Babel is stable, well established 
technology.  No offense intended to Babel, but its Javascript lexer has 
only 89 lines of tests, so by your line-count criteria, Babel isn't 
ready to depend on either.  And an examination of the code shows that it 
will mishandle at least one case (the regex /[/]/).  Known bugs in my 
patch: 0, known bugs in Babel: 1.


But this is an odd debate, there are really three solutions to evaluate: 
the existing code, my patch, and Babel, and by any yardstick, the 
existing code is badly broken.


I have no idea how widely used Babel is, but it can't be as wide as the 
Gnu gettext utilities, so we'd have to evaluate those other components 
of Babel as well.  Are there edge cases in .po and .mo files that it 
doesn't handle properly?  I have no idea.


Keep in mind that the proposal is not to include Babel, but to depend on 
it as a prerequisite, which means we are stuck in the same situation we 
are with gettext: it can change independently of Django, and new 
versions can introduce new bad behavior.  That's one of the reasons we 
have a bad problem today: gettext changed from 0.17 to 0.18, and 
exacerbated the hack.  Babel has the advantage that it is pure Python, 
so it is both more installable than gettext, and is more readable for 
us.  It also has the advantage that it isn't based on a hack, but that 
doesn't mean it performs flawlessly.


BTW: the Perl-based lexer is not broken in

Re: Fixing makemessages for Javascript

2011-04-14 Thread Ned Batchelder


On 4/14/2011 10:30 AM, Jannis Leidel wrote:

On 14.04.2011, at 15:54, Ned Batchelder wrote:


On 4/14/2011 9:26 AM, Jannis Leidel wrote:

On 14.04.2011, at 14:41, Ned Batchelder wrote:


Does anyone else have any opinions on a direction forward to fix this problem?  
At the very least I'd like to make a doc patch for 1.3.1 that explains the 
fragility.

Yeah, a doc patch sounds like a good plan.

Jannis


Frankly, I'm disappointed by this approach.  Shall I draft the paragraph that says 
roughly, "This feature of Django doesn't work and will fail silently.  Please find a 
third-party alternative."?

This is hardly a new issue (see the age of the tickets), so we should only 
clarify that using the recent gettext (0.18.1.1) won't work.

Having said that, are you at all interested in working on the Babel-based 
solution?

Jannis

Precisely because of the age of the tickets, we know this isn't limited 
to recent gettext.  The problem has been exacerbated by recent gettext, 
but in fact, it's been a problem with older gettext as well.  
Apostrophes in line comments have always caused issues, for example.  
There is no version of gettext that works well for Javascript.  This 
feature of Django is simply broken.  I expect as more users have gettext 
0.18, we'll hear more about it, as you can already see from the recent 
uptick in activity on these old tickets.


I don't know what to write in the docs to explain to users that this 
doesn't work in Django.  1.3 will be used for a very long time by many 
people, I'd love to be able to tell them that Django can support 
localization of Javascript text out of the box.


Are there any other committers (or even a BDFL) with an opinion about 
this?  We've gathered a -1 from a committer and three +1 (including 
mine) from the community.


Having said all that, I am very interested in making this work well in 
Django.  If I can help with Babel, let me know.


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-04-14 Thread Ned Batchelder




On 4/14/2011 9:26 AM, Jannis Leidel wrote:

On 14.04.2011, at 14:41, Ned Batchelder wrote:


Does anyone else have any opinions on a direction forward to fix this problem?  
At the very least I'd like to make a doc patch for 1.3.1 that explains the 
fragility.

Yeah, a doc patch sounds like a good plan.

Jannis

Frankly, I'm disappointed by this approach.  Shall I draft the paragraph 
that says roughly, "This feature of Django doesn't work and will fail 
silently.  Please find a third-party alternative."?


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-04-14 Thread Ned Batchelder

Does anyone else have any opinions on a direction forward to fix this 
problem?  At the very least I'd like to make a doc patch for 1.3.1 that 
explains the fragility.


--Ned.

On 4/9/2011 1:02 PM, Ned Batchelder wrote:

On 4/9/2011 11:57 AM, Jannis Leidel wrote:

On 09.04.2011, at 17:32, Ned Batchelder wrote:


On 4/9/2011 10:43 AM, Jannis Leidel wrote:

On 09.04.2011, at 16:14, Ned Batchelder wrote:

I've created two patches implementing this strategy, and attached 
them to ticket http://code.djangoproject.com/ticket/7704.
Thanks Ned, but I'm a bit confused, I thought we agreed that Babel 
should be the way to go, since adding an own JavaScript lexer into 
Django would further move us away from our goal to get rid of the 
dependency on xgettext.


Jannis
I understood that a few people including you expressed a preference 
for integrating Babel. But I don't see any motion toward doing it, 
and it brings its own questions, such as the relationship between 
Django and Babel.  Forgive me if I'm wrong, but it seemed like 
switching from xgettext to Babel was not a simple proposition, and 
had its detractors.
We've just been out with 1.3 for a few weeks, so there is no big 
surprise we haven't produced any substancial patches for moving away 
from xgettext. But that doesn't mean that I (and it seems a few 
others) haven't researched the bits needed to make it happen. As for 
the "detractors", I'm not even sure what you mean by that.
I thought I had read some issues about integrating Babel, but I can't 
find them now.  It must have been a figment of my lexer-fevered brain, 
my mistake.  What are the bits needed to make it happen?


For example, could we switch to Babel for 1.3.1?  I'd very much like 
to have this headache gone as soon as possible.  The lexer I've 
attached to the ticket works well, at least I'd like to hear from 
someone who believes it doesn't.
No, Babel can't be adopted in the 1.3.X release branch, just like 
your JavaScript lexer.
Even if Babel is the solution for 1.4, I don't understand why my patch 
can't be applied to 1.3.x?  It's well tested, and clearly works better 
than the code in 1.3 now.  It's transparent to the user, except it 
isn't baffling like today's code.  From my point of view, makemessages 
simply doesn't work for Javascript files.  It's a frustrating process 
that usually ends with twisting your Javascript sources to meet the 
needs of a Perl parser, but without understanding that that's what 
you're doing.


I understand the desire to move away from xgettext, but it doesn't 
seem to be happening.  No one had claimed the three open tickets 
about this problem, for instance.
How can you say it's not happening? I clearly made a statement about 
my committement for Babel adoption in the current release cycle.
I was basing that on the fact that the tickets weren't claimed.  
Again, I don't mean to antagonize anyone, I just want this to be fixed.


--Ned.



--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-04-09 Thread Ned Batchelder


On 4/9/2011 11:57 AM, Jannis Leidel wrote:

On 09.04.2011, at 17:32, Ned Batchelder wrote:


On 4/9/2011 10:43 AM, Jannis Leidel wrote:

On 09.04.2011, at 16:14, Ned Batchelder wrote:


I've created two patches implementing this strategy, and attached them to 
ticket http://code.djangoproject.com/ticket/7704.

Thanks Ned, but I'm a bit confused, I thought we agreed that Babel should be 
the way to go, since adding an own JavaScript lexer into Django would further 
move us away from our goal to get rid of the dependency on xgettext.

Jannis

I understood that a few people including you expressed a preference for 
integrating Babel. But I don't see any motion toward doing it, and it brings 
its own questions, such as the relationship between Django and Babel.  Forgive 
me if I'm wrong, but it seemed like switching from xgettext to Babel was not a 
simple proposition, and had its detractors.

We've just been out with 1.3 for a few weeks, so there is no big surprise we haven't 
produced any substancial patches for moving away from xgettext. But that doesn't mean 
that I (and it seems a few others) haven't researched the bits needed to make it happen. 
As for the "detractors", I'm not even sure what you mean by that.
I thought I had read some issues about integrating Babel, but I can't 
find them now.  It must have been a figment of my lexer-fevered brain, 
my mistake.  What are the bits needed to make it happen?



For example, could we switch to Babel for 1.3.1?  I'd very much like to have 
this headache gone as soon as possible.  The lexer I've attached to the ticket 
works well, at least I'd like to hear from someone who believes it doesn't.

No, Babel can't be adopted in the 1.3.X release branch, just like your 
JavaScript lexer.
Even if Babel is the solution for 1.4, I don't understand why my patch 
can't be applied to 1.3.x?  It's well tested, and clearly works better 
than the code in 1.3 now.  It's transparent to the user, except it isn't 
baffling like today's code.  From my point of view, makemessages simply 
doesn't work for Javascript files.  It's a frustrating process that 
usually ends with twisting your Javascript sources to meet the needs of 
a Perl parser, but without understanding that that's what you're doing.



I understand the desire to move away from xgettext, but it doesn't seem to be 
happening.  No one had claimed the three open tickets about this problem, for 
instance.

How can you say it's not happening? I clearly made a statement about my 
committement for Babel adoption in the current release cycle.
I was basing that on the fact that the tickets weren't claimed.  Again, 
I don't mean to antagonize anyone, I just want this to be fixed.


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-04-09 Thread Ned Batchelder


On 4/9/2011 10:43 AM, Jannis Leidel wrote:

On 09.04.2011, at 16:14, Ned Batchelder wrote:


I've created two patches implementing this strategy, and attached them to 
ticket http://code.djangoproject.com/ticket/7704.

Thanks Ned, but I'm a bit confused, I thought we agreed that Babel should be 
the way to go, since adding an own JavaScript lexer into Django would further 
move us away from our goal to get rid of the dependency on xgettext.

Jannis
I understood that a few people including you expressed a preference for 
integrating Babel. But I don't see any motion toward doing it, and it 
brings its own questions, such as the relationship between Django and 
Babel.  Forgive me if I'm wrong, but it seemed like switching from 
xgettext to Babel was not a simple proposition, and had its detractors.


For example, could we switch to Babel for 1.3.1?  I'd very much like to 
have this headache gone as soon as possible.  The lexer I've attached to 
the ticket works well, at least I'd like to hear from someone who 
believes it doesn't.  I understand the desire to move away from 
xgettext, but it doesn't seem to be happening.  No one had claimed the 
three open tickets about this problem, for instance.


I apologize if I'm being brash or impatient here.  What's the right way 
forward?


--Ned.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-04-09 Thread Ned Batchelder

I've created two patches implementing this strategy, and attached them 
to ticket http://code.djangoproject.com/ticket/7704.


--Ned.

On 4/4/2011 5:15 PM, Ned Batchelder wrote:
Last week I re-encountered the problems with using makemessages on 
Javascript files, and lost a couple of half-days to trying to figure 
out why some of my translatable messages weren't being found and 
deposited into my .po files.  After fully understanding the extent of 
Django's current hack, I decided to take a stab at providing a better 
solution.


Background: today, Javascript source files are parsed for messages by 
running a "pythonize" regex over them, and giving the resulting text 
to xgettext, claiming it is Perl.  The pythonize regex simply changes 
any //-style comment on its own line into a #-style comment.  This 
strange accommodation leaves a great deal of valid Javascript syntax 
in place to confuse the Perl parser in xgettext.  As a result, 
seemingly innocuous Javascript will result in lost messages:


gettext("xyzzy 1");
var x = y;
gettext("xyzzy 2");
var x = z;
gettext("xyzzy 3");

In this sample, messages 1 and 3 are found, and message 2 is not, 
because y;ABC;abc; is valid Perl for a transliteration operator.  
Digging into this, every time I thought I finally understood the full 
complexity of the brokenness, another case would pop up that didn't 
make sense.  The full horror of Perl syntax 
(http://perldoc.perl.org/perlop.html#Quote-and-Quote-like-Operators , 
for example) means that it is very difficult to treat non-Perl code as 
Perl and expect everything to be OK.  This is polyglot programming at 
its worst.


This needs to be fixed.  To that end, I've written a Javascript lexer 
(https://bitbucket.org/ned/jslex) with the goal of using it to 
pre-process Javascript into a form more suitable for xgettext.  My 
understanding of why we claim Javascript is Perl is that Perl has 
regex literals like Javascript does, and so xgettext stands the best 
chance of parsing Javascript as Perl.  Clearly that's not working 
well.  My solution would instead remove the regex literals from the 
Javascript, and then have xgettext treat it as C.


I have a few questions you can help me with:

1. Is this the best path forward?  Ideally xgettext would support 
Javascript directly. There's code out there to add Javascript to 
xgettext, but I don't know what shape that code is in, or if it's 
reasonable to expect Django installations to use bleeding-edge 
xgettext.  Is there some better solution that someone is pursuing?


2. Is there some other badness that will bite us if we tell xgettext 
that the modified Javascript is C?  With a full Javascript lexer, I 
feel pretty confident that we could solve issues if they do come up, 
but I'd like to know now what they are.


3. I know that lexing Javascript is tricky.  I need help finding 
diabolical test cases for my lexer (https://bitbucket.org/ned/jslex).  
Anyone care to come up with some Javascript source that it can't 
properly find the regex literals in?


BTW: This would close tickets #7704, #14045, #15331, and #15495.

--Ned.

--
You received this message because you are subscribed to the Google 
Groups "Django developers" group.

To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Fixing makemessages for Javascript

2011-04-05 Thread Ned Batchelder

Jannis and Łukasz have both suggested the same thing: use Babel instead 
of xgettext. I understand why: it's a more complete solution than what I 
have proposed, which is at heart still a way to trick xgettext into 
parsing source code it doesn't natively understand.


I have no experience with Babel, so I don't know what work lies ahead to 
integrate it with Django. I have used my code in makemessages, and it 
works well.


Questions now include:

1) What can we get done in 1.3.1? Is integrating Babel something that 
would have to wait for 1.4?


2) Who is the best expert on Babel and Django that could comment on the 
work needed?


3) Are there other opinions about the two paths forward? Are there other 
options?


I would like very much to get this problem solved.

--Ned.

On 4/4/2011 6:42 PM, Jannis Leidel wrote:

On 04.04.2011, at 23:15, Ned Batchelder wrote:


Last week I re-encountered the problems with using makemessages on Javascript 
files, and lost a couple of half-days to trying to figure out why some of my 
translatable messages weren't being found and deposited into my .po files.  
After fully understanding the extent of Django's current hack, I decided to 
take a stab at providing a better solution.

Background: today, Javascript source files are parsed for messages by running a 
"pythonize" regex over them, and giving the resulting text to xgettext, 
claiming it is Perl.  The pythonize regex simply changes any //-style comment on its own 
line into a #-style comment.  This strange accommodation leaves a great deal of valid 
Javascript syntax in place to confuse the Perl parser in xgettext.  As a result, 
seemingly innocuous Javascript will result in lost messages:
gettext("xyzzy 1");
var x = y;
gettext("xyzzy 2");
var x = z;
gettext("xyzzy 3");
In this sample, messages 1 and 3 are found, and message 2 is not, because 
y;ABC;abc; is valid Perl for a transliteration operator.  Digging into 
this, every time I thought I finally understood the full complexity of the 
brokenness, another case would pop up that didn't make sense.  The full 
horror of Perl syntax 
(http://perldoc.perl.org/perlop.html#Quote-and-Quote-like-Operators , for 
example) means that it is very difficult to treat non-Perl code as Perl and 
expect everything to be OK.  This is polyglot programming at its worst.

This needs to be fixed.  To that end, I've written a Javascript lexer 
(https://bitbucket.org/ned/jslex) with the goal of using it to pre-process 
Javascript into a form more suitable for xgettext.  My understanding of why we 
claim Javascript is Perl is that Perl has regex literals like Javascript does, 
and so xgettext stands the best chance of parsing Javascript as Perl.  Clearly 
that's not working well.  My solution would instead remove the regex literals 
from the Javascript, and then have xgettext treat it as C.

Thanks Ned, I meant the post about this issue here after 1.3, since we also 
talked about this during the Pycon sprint, especially since we seem to have hit 
a few more problems with the recent gettext 0.18.1.1 (such as a seemingly 
stricter Perl lexer) -- which I encountered while I applied the final 
translation updates right before 1.3 but didn't have time to investigate yet. 
The bottom line is that I think we should rethink the way we look for 
translateable strings instead of working around the limitations of xgettext.


1. Is this the best path forward?  Ideally xgettext would support Javascript 
directly. There's code out there to add Javascript to xgettext, but I don't 
know what shape that code is in, or if it's reasonable to expect Django 
installations to use bleeding-edge xgettext.  Is there some better solution 
that someone is pursuing?

We can't really expect Django users to upgrade to the most recent (or even an 
unreleased) version of gettext, We've bumped the minimum required version in 
Django 1.2 to 0.15 once all OSes were covered with installers. Which made me 
talk to Armin Ronacher about using Babel instead of GNU gettext, since it has a 
JavaScript lexer and is in use in Sphinx and Trac. [1] In that sense, I 
wholeheartedly encourage you to take a stab at it for 1.4 -- if you think 
that's a good idea.

Having a Python based library (assuming it works similarly) seems like a better 
fit to Django than relying on a C program.


2. Is there some other badness that will bite us if we tell xgettext that the 
modified Javascript is C?  With a full Javascript lexer, I feel pretty 
confident that we could solve issues if they do come up, but I'd like to know 
now what they are.

I feel this is much better solved once and fall all than to keep misusing 
xgettext.

Jannis

1: http://babel.edgewall.org/browser/trunk/babel/messages/jslexer.py




--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To

Re: Fixing makemessages for Javascript

2011-04-04 Thread Ned Batchelder


On 4/4/2011 5:45 PM, Łukasz Rekucki wrote:

On 4 April 2011 23:15, Ned Batchelder<n...@nedbatchelder.com>  wrote:

I have a few questions you can help me with:

1. Is this the best path forward?  Ideally xgettext would support Javascript
directly. There's code out there to add Javascript to xgettext, but I don't
know what shape that code is in, or if it's reasonable to expect Django
installations to use bleeding-edge xgettext.  Is there some better solution
that someone is pursuing?

It would be best to have xgettext allow pluggable parsers or/and
rewritten in Python. Oh wait, it's called Babel ;). I was planning to
refactor the `makemessages` command to enable custom parsers (My
motivation is having a bunch of client-side templates written in a
variant of Mustache or jQuery Templates). As babel already has an
interface for that, integrating it into Django would be cool. Sure,
it's a dependancy, but so is xgettext.
I don't understand yet how Babel fits into the ecosystem, but if it's a 
better xgettext, then that might be the way to go.  I see a jslexer.py 
in the source, I wish I'd seen that a few days ago! :)

2. Is there some other badness that will bite us if we tell xgettext that
the modified Javascript is C?  With a full Javascript lexer, I feel pretty
confident that we could solve issues if they do come up, but I'd like to
know now what they are.

IMHO, it would be best to leave only gettext() calls and pad them with
comments, so that line numbers match (the template converter does
something similar, by replacing all other stuff with string like
). You can then choose C, Python or whatever :)

I saw your comment to that effect on one of the tickets. That approach 
would probably also work.  All of these approaches presuppose accurate 
tokenization of the Javascript, which we can now do.


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Fixing makemessages for Javascript

2011-04-04 Thread Ned Batchelder

Last week I re-encountered the problems with using makemessages on 
Javascript files, and lost a couple of half-days to trying to figure out 
why some of my translatable messages weren't being found and deposited 
into my .po files.  After fully understanding the extent of Django's 
current hack, I decided to take a stab at providing a better solution.


Background: today, Javascript source files are parsed for messages by 
running a "pythonize" regex over them, and giving the resulting text to 
xgettext, claiming it is Perl.  The pythonize regex simply changes any 
//-style comment on its own line into a #-style comment.  This strange 
accommodation leaves a great deal of valid Javascript syntax in place to 
confuse the Perl parser in xgettext.  As a result, seemingly innocuous 
Javascript will result in lost messages:


   gettext("xyzzy 1");
   var x = y;
   gettext("xyzzy 2");
   var x = z;
   gettext("xyzzy 3");

In this sample, messages 1 and 3 are found, and message 2 is not, 
because y;ABC;abc; is valid Perl for a transliteration operator.  
Digging into this, every time I thought I finally understood the full 
complexity of the brokenness, another case would pop up that didn't make 
sense.  The full horror of Perl syntax 
(http://perldoc.perl.org/perlop.html#Quote-and-Quote-like-Operators , 
for example) means that it is very difficult to treat non-Perl code as 
Perl and expect everything to be OK.  This is polyglot programming at 
its worst.


This needs to be fixed.  To that end, I've written a Javascript lexer 
(https://bitbucket.org/ned/jslex) with the goal of using it to 
pre-process Javascript into a form more suitable for xgettext.  My 
understanding of why we claim Javascript is Perl is that Perl has regex 
literals like Javascript does, and so xgettext stands the best chance of 
parsing Javascript as Perl.  Clearly that's not working well.  My 
solution would instead remove the regex literals from the Javascript, 
and then have xgettext treat it as C.


I have a few questions you can help me with:

1. Is this the best path forward?  Ideally xgettext would support 
Javascript directly. There's code out there to add Javascript to 
xgettext, but I don't know what shape that code is in, or if it's 
reasonable to expect Django installations to use bleeding-edge 
xgettext.  Is there some better solution that someone is pursuing?


2. Is there some other badness that will bite us if we tell xgettext 
that the modified Javascript is C?  With a full Javascript lexer, I feel 
pretty confident that we could solve issues if they do come up, but I'd 
like to know now what they are.


3. I know that lexing Javascript is tricky.  I need help finding 
diabolical test cases for my lexer (https://bitbucket.org/ned/jslex).  
Anyone care to come up with some Javascript source that it can't 
properly find the regex literals in?


BTW: This would close tickets #7704, #14045, #15331, and #15495.

--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Project-wide cache prefix (low-level API)

2010-08-04 Thread Ned Batchelder


On 8/4/2010 10:57 AM, Jacob Kaplan-Moss wrote:

On Wed, Aug 4, 2010 at 8:06 AM, Byron  wrote:
   

Updated the patch http://code.djangoproject.com/ticket/13795
 


* Have you considered supporting "versioning" of keys to help with cache
   invalidation? Eric Florenzano has been doing some interesting
   experimenting along those lines in django-newcache
   (http://github.com/ericflo/django-newcache); you may want to look at the
   code and/or talk to him about working some of his ideas back into your
   key prefix proposal. At the very least, any changes we make to the
   caching backend should allow the sorts of things he's doing to keep
   working; ideally we should make those sorts of changes really easy to
   make.

   
Couldn't versioning be handled by setting CACHE_KEY_PREFIX to include 
some varying data like the svn revision?  We've got a cache key prefix 
hacked into our work environment, and I ensure my dev machine never gets 
stale cache data like this:


   import time
   CACHE_KEY_PREFIX = "dev-ned-%s" % time.time()

This uses the start time of the dev server as part of the cache key so 
each invocation gets fresh data.  In production, you'd use something 
different, but this illustrates the point.  Or is there some more 
elaborate versioning being considered here?


--Ned.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: managing javascript and css resources

2010-04-22 Thread Ned Batchelder

FWIW, I just started using django-compress 
(http://code.google.com/p/django-compress/) which works precisely this 
way.  It has pluggable compressors, control over the versioning of the 
combined files, and so on.


--Ned.

Gabriel Hurley wrote:

I like the idea of having these "bundles" or "stacks" for media. Just
thinking out loud here, if there were a compression engine in play
that could do concatenation as well as minification you could have a
useful syntax for ordered combinations of scripts similar to how
ModelAdmin's fieldsets work (using a dictionary of nested tuples).

A quick example:

scripts = {
"head": (
"media/js/script_to_be_loaded_first.js",
),
"tail": (
"media/js/script_that_shouldn't_be_concatenated.js", ("media/
js/concatenate_me_1.js", "media/js/concatenate_me_2.js"),
)
}

That would give you the benefit of automated concatenation while
somewhat alleviating the problem Kevin Howerton pointed out about it
best being done by hand.

That's my thought for the moment.

- Gabriel

On Apr 21, 9:18 am, Owen Nelson  wrote:
  

I've been thinking about this ever since I learned that django's admin
was going to be using jQuery, but I admit I didn't really consider it
that important until recently (building sites against 1.2-beta).

I know now is not a fantastic time to be talking about features, but
this is something I'd enjoy working on (personally), and I am just
hoping to get a little feedback regarding the design, and how it might
fit in with everything else going on in django's guts
(philosophically).  I also understand that this isn't something for near
versions of django, but rather the distant future.

Here's where the pin dropped:
* The jQuery used by the admin site is conjured using the no conflict
option and is relegated/isolated to it's own namespace (excellent).
* There are many projects/apps out there that also rely on jQuery - they
also "bundle" the framework, but may not be quite so quick to play nice.

In my case, I noticed that when I added a few jQuery-enhanced form
widgets to a form in my admin site, it resulted in 3 instances of the
framework being sourced in the document head.  Although, this is
actually ok for the most part in terms of operation (so long as the
scripted events that come with each widget are bound to the dom before
the plugin of origin gets wiped by the next sourcing of jQuery), it's
far from ideal.

There are a myriad of ways to skin this cat, be it a simple
recommendation to adopt  the use of  django's jQuery and a template tag
to provide access to the appropriate script tag, or something more
structured/engineered/formal 

My goal would be to provide app developers with scaffolding to add
javascript/css resources to the rendered view in a non-competitive way.
I'm thinking in terms of a template tag that works along the lines of {%
include %}, but for script and link tags, allowing folks to add scripts
with an optional way to influence the position in the "stack".  A
similar interface would also be provided for form media, and perhaps
some kind of helper or shortcut for ease the addition of these scripts
from our view functions.

I understand that Django has historically been
anti-javascript-framework-blessing, and I'm wondering if opening this
can of worms would mean having to incorporate some kind of a pluggable
backend system (for working with different frameworks, or multiple
frameworks at a time) - something I've briefly considered, but started
foaming at the mouth as a result.
I've also considered the fact that the reaction here might be, "just
don't - leave it all in the individual's hands".

In closing... don't hit me!

Regards,
Owen Nelson

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group 
athttp://groups.google.com/group/django-developers?hl=en.



  


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal for 1.2: Improved URL system

2009-10-17 Thread Ned Batchelder

This seems to me to be a perfect candidate for implementation outside of 
core.  Write a facility that takes your simpler url patterns, and 
creates standard Django urlpatterns from them.  Publish it.  If it 
catches on, then we can discuss what future version of Django might 
include it as core.

BTW: I've also thought that regular expressions are great for their 
power, but generally overkill for url patterns.

--Ned.

Leaf wrote:
> Okay, I guess I can live with that. Maybe, if I can write a patch
> implementing this in a completely backwards-compatible way, I'll
> propose it again for a later version (earlier in the release cycle).
>
> Though I disagree with your characterization of it as "what looks like
> regular expressions with names." It would be compiled down to regular
> expressions, yes, but it's not really intended to be just a wrapper
> over regular expressions. (I wrote one of those in about three
> minutes.) Maybe if you looked at Werkzeug's routing source code, you'd
> see what I mean. I personally like it because it's cleaner, both using
> it from the front end and in the source.
>
> -- LeafStorm
>
> On Oct 17, 7:00 pm, James Bennett  wrote:
>   
>> On Sat, Oct 17, 2009 at 10:29 AM, Leaf  wrote:
>> 
>>> Sorry to propose this right up against the voting deadline for 1.2
>>> features, but it's one of the things that has always bugged me about
>>> Django and I would really like to see this in 1.2.
>>>   
>> -1.
>>
>> Replacing regular expressions with... well, what looks like regular
>> expressions with names doesn't make much sense, and even if it did
>> we're not going to change a major component like this in the middle of
>> a stable release line.
>>
>> --
>> "Bureaucrat Conrad, you are technically correct -- the best kind of correct."
>> 
> >
>
>   

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal for 1.2: Dumber email validation

2009-10-10 Thread Ned Batchelder


+1

http://nedbatchelder.com/blog/200908/humane_email_validation.html

I was going to kibbitz on the fix (removing a single * would have 
sufficed), and realized we were once again in the quagmire of email 
regex validation.

--Ned.

James Bennett wrote:
> In light of yesterday's security issue, I'd like to propose that we
> significantly dumb down the regex Django uses to validate email
> addresses.
>
> Currently, the regex we use covers many common cases, but comes
> nowhere near covering the entire spectrum of addresses allowed by the
> RFC; several tickets are open regarding this. Trying to cover more of
> the RFC is possible, although supporting all valid email addresses is
> not (various regexes claim to do this, but full coverage is impossible
> -- the RFC is flexible enough WRT things like nested comments that I'm
> fairly certain no single regex can handle them all), and -- as we've
> seen -- attempts to cover a broader chunk of the RFC can introduce
> issues with performance.
>
> So what I'd like to propose is that EmailField essentially check that
> the value contains an '@', and a '.' somewhere after it. This will
> cover most addresses that are likely to be in actual use, and various
> confirmation processes can be used to rule out any invalid addresses
> which happen to slip through that. Meanwhile, people who want to
> support comments inside a bang path or other such exotic beasts can
> simply write their own regex for it and tell a form to use that
> instead.
>
>
>
>
>   

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: Tutorial Refresh

2009-10-10 Thread Ned Batchelder

Russell Keith-Magee wrote:
> On Sat, Oct 10, 2009 at 6:55 AM, Rob Hudson  wrote:
>   
>> I, too, like the idea of a conference site.  It fills a void and
>> sounds useful for upcoming conferences.  I wasn't too crazy about the
>> blog idea, and was convinced away from the snippets idea.  So shall we
>> call it a conference site and move on?  :)
>> 
>
> A conference site it is then.
>
> One piece of guidance, especially if you're aiming for the final site
> to be used for DjangoCon. Keep in mind that a tutorial needs to be
> simple at the start. However, a real site for a conference will be a
> lot more complex. There is a delicate balance that will need to be
> made between "useful tutorial example" and "useful real site".
>
>   
My strong feeling is that these two goals will quickly become impossible 
to reconcile.  I think the idea of a conference site is a good one 
(everyone will understand the problem domain, lots of interesting 
avenues to explore, it's not yet-another-blog), but aiming for it to be 
the actual site used for DjangoCon will not work in the long run. 

The tutorial is extremely important.  It will be the first part of the 
docs read by 98% of new users.  Don't complicate it by tying it to 
DjangoCon.  This thread has already seen requests for features that will 
be great for real use, but would probably be too much to put into a 
tutorial.  While it would be very cool to have the two sites be just one 
site, I think it will either create an overgrown underexplained tutorial 
as DjangoCon adds features needed for a real-world conference site, or a 
simplistic toy DjangoCon site because the tutorial needs to ensure that 
everything is clean, understandable, and accompanied by clear prose.

Why not serve just one master well, and have the tutorial be purely 
about exposition and pedagogy?

Am I being too pessimistic?

--Ned.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Adding an option to re-test only failed tests

2009-10-02 Thread Ned Batchelder

Rob Madole wrote:
>> From the point of view of encouraging the usage of nose, either would
>> work fine. I think this is fits in to the conversation at DjangoCon
>> about how we should go about encouraging Django users to explore the
>> wider Python ecosystem. The important thing is that we can have some
>> official (or semi-official) documentation somewhere saying "to access
>> enhanced test running commands, install nose and drop this string in
>> to your TEST_RUNNER setting." The string itself could point at
>> dango.something or nose.something, the important thing from my point
>> of view is that they don't have to grab a test runner script from
>> another third party, then figure out where to put it within their
>> project. If nose don't want to ship the test runner, I'd be fine with
>> putting it in django.contrib somewhere.
>> 
>
> It's hard to get people to write tests.  It's hard to get them to
> document their work.  I wouldn't care if it's in Nose or Django that
> has the test runner, just as long as doing an "easy_install nose" and
> changing the TEST_RUNNER is all a user has to do to make the switch.
> I just can't imagine the Nose guys including a Django test runner in
> their base install.  Maybe I'm wrong.
>
>   
I've been wrestling with a similar issue in coverage.py:  As I update 
coverage.py and break the nose coverage plugin, how should it get 
fixed?  The nose developers don't seem to have the cycles to update the 
coverage plugin today.  I'm coming around to the conclusion that 
coverage.py should ship the nose coverage plugin, because coverage.py is 
changing faster than the nose plugin API is. 

I would think the same logic applies to Django.  Nose needs to work with 
lots of different projects, so they can't own the Django details, since 
by that logic they'd also own the TurboGears logic, the Pylons logic, 
the Twisted logic, etc...  But the Nose plugin API isn't changing much 
now, so Django could include a nose plugin which would only have to 
change when Django details change.  And we'd get all sorts of nose 
features without having to implement them in the Django code.

--Ned.
http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: [gsoc2009-testing] Windmill Runners Kicking it up a Notch

2009-06-29 Thread Ned Batchelder

the elaborate charade we play (conning
> windmill to play with us) is reliable for 3rd party
> applications as well. 
>
> The branch isn't really ready for testing yet, but it has been
> known to work. And Eric has kindly thrown up a coverage report!
>
> http://media.ericholscher.com/django_coverage/
> -- 
> Kevin Kubasik
> http://kubasik.net/blog
>
>
>
> Just a small note, but there seems to be an issue with the
> coverage, in that any module level statements aren't reported as
> being executed, such as imports, function definitions, or class
> definitions.  That might be an issue with whatever does the
> coverage report though.
>
> Alex
>
> -- 
> "I disapprove of what you say, but I will defend to the death your
> right to say it." --Voltaire
> "The people's good is the highest law."--Cicero
>
>
>
>
>
> -- 
> Kevin Kubasik
> http://kubasik.net/blog
>
> >

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Posting to the wrong list (was: Re: Need Django Developer urgent)

2009-05-08 Thread Ned Batchelder

Add the word "core" to make the first sentence, "Discussion group for 
Django core developers".

Not that that will make them read the description, but...

--Ned.

Jacob Kaplan-Moss wrote:
> On Fri, May 8, 2009 at 12:59 PM, Joshua Partogi <joshua.j...@gmail.com> wrote:
>   
>> Some people get confused and thought that this list is for people that
>> develop apps with django :-D
>> 
>
> Those people might want to take the time to read the list description
> (http://groups.google.com/group/django-developers) that they see when
> signing up, which reads:
>
> """
> Discussion group for Django developers. This group is used for
> discussion of developing Django itself, not user questions; Please use
> django-users for issues regarding using the framework, questions for
> the Django user community, outreach, etc.
> """
>
> If you've got any suggestions of ways to make that more clear -- other
> than renaming the list, which isn't going to happen -- I'd love to
> hear 'em.
>
> Jacob
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Test integration with coverage

2009-04-09 Thread Ned Batchelder

George Song wrote:
> On 4/7/2009 11:25 AM, Kevin Kubasik wrote:
>   
>> I actually proposed this as part of my GSoC project. I think there is
>> enough interest that basic coverage support could be seen in core.
>> 
>
> I agree and have had some quick emails with Jacob about this. I think
> there's a way to make this a stand-alone package as well as integrate
> it
> into 1.2. That way people who want to use it today, can.
>
>   
>> My research turned up figleaf, which seemed like a better fit (but
>> its an external dependency, and I'm not sure of the django policy on
>> this, especially when coverage.py is already pretty good ).
>> 
>
> Well, ``coverage.py`` is a third-party tool as well:
> <http://nedbatchelder.com/code/modules/coverage.html>
>
> I think Django is becoming more accepting of third-party libraries,
> why
> re-invent the wheel?
>
> I'm interested to hear why you think ``figleaf`` is a better fit than
> ``coverage``?
>
> --
> George
>   
George, I'd love to hear about your changes to coverage.py.  I've been 
working on a major refactoring recently: 
http://bitbucket.org/ned/coveragepy/ .  I'd also be glad to help any way 
I can to get coverage better integrated into Django. 

--Ned.
http://nedbatchelder.com/

> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Dropping Python 2.3 compatibility for Django 1.1

2008-11-25 Thread Ned Batchelder

One way to collect feedback would be to make one small change to the 
code that would require 2.4, and ship 1.1 that way.  Then we'd hear from 
people who really couldn't run 1.1, but we haven't made too large a 
change yet, so if we wanted to re-enable them we could.  I realize this 
means putting off being able to use 2.4 syntax.  The only reason to take 
this path is if the devs want to be conservative and get a real data 
point before completely moving away from 2.3.

--Ned.
http://nedbatchelder.com

[EMAIL PROTECTED] wrote:
> Sounds double plus good(+1) from me.  That being said, it's been said
> before that Djagno-dev, even if 20 people are vocally in favor of
> something, is a tiny fraction of all the people using Django, is there
> perhaps a better/more objective way of collecting feedback(the
> obviously, perhaps only, argument against is that some people don't
> have access to it, the only question is that a sizeable portion of
> Django users).
>
> alex
>
> On Nov 25, 12:33 pm, Ivan Sagalaev <[EMAIL PROTECTED]> wrote:
>   
>> Ludvig Ericson wrote:
>> 
>>> On Nov 25, 2008, at 18:08, Jacob Kaplan-Moss wrote:
>>>   
>>>> I'd like to officially drop Python 2.3 support in Django 1.1. Discuss.
>>>> 
>>> Oh god please, YES! Gimme my decorator syntax sugar, oh yeah.
>>>   
>> ... and generator expressions, too!
>>
>> +1
>> 
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: Optional {% default %} clause for the {% for %} template tag

2008-10-29 Thread Ned Batchelder

Yes, you are right!

--Ned.

Calvin Spealman wrote:
> The else clause executes unless a loop is broken (that is, with the 
> break or obviously by raising an exception), and executes after a loop 
> ends normally.
>
> >>> for i in "abc":
> ...   pass
> ... else:
> ...   print "else"
> else
> >>>
>
> On Wed, Oct 29, 2008 at 8:32 PM, Ned Batchelder <[EMAIL PROTECTED] 
> <mailto:[EMAIL PROTECTED]>> wrote:
>
> But this does operate the same as the Python for/else, no?
>
> >>> for i in []:
> ...  print "boo"
> ... else:
> ...  print "foo"
> ...
> foo
> >>>
>
> --Ned.
> http://nedbatchelder.com
>
>
> Calvin Spealman wrote:
>> On Tue, Oct 28, 2008 at 8:18 PM, oggie rob
>> <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
>>
>>
>> > {% for item in grocery_list %}
>> >   {{ item }}
>> > {% default %}
>> >   Nothing to buy.
>> > {% endfor %} 
>>
>>
>> Please, though - use {% else %}. Its totally clear what its
>> referring
>> to and else doesn't mean squat unless you see what the if
>> (and in this
>> case, for) test is anyway, so I don't think this would be
>> confusing
>> (after all, this isn't python).
>> (Also, if you want to avoid confusion don't use a keyword that is
>> located within another language's looping construct :)
>>
>>
>> Please dont use else, because {%for%} matches python's for loop
>> and that supports an else clause which does not operate like
>> this. If the same keyword is used, it should behave the same.
>>
>>
>> -- 
>> Read my blog! I depend on your acceptance of my opinion! I am
>> interesting!
>> http://techblog.ironfroggy.com/
>> Follow me if you're into that sort of thing:
>> http://www.twitter.com/ironfroggy
>>
>>
>
> -- 
> Ned Batchelder, http://nedbatchelder.com
> 
>
>
>
>
>
>
> -- 
> Read my blog! I depend on your acceptance of my opinion! I am interesting!
> http://techblog.ironfroggy.com/
> Follow me if you're into that sort of thing: 
> http://www.twitter.com/ironfroggy
>
> >

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: Optional {% default %} clause for the {% for %} template tag

2008-10-29 Thread Ned Batchelder

But this does operate the same as the Python for/else, no?

 >>> for i in []:
...  print "boo"
... else:
...  print "foo"
...
foo
 >>>

--Ned.
http://nedbatchelder.com

Calvin Spealman wrote:
> On Tue, Oct 28, 2008 at 8:18 PM, oggie rob <[EMAIL PROTECTED] 
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>
> > {% for item in grocery_list %}
> >   {{ item }}
> > {% default %}
> >   Nothing to buy.
> > {% endfor %} 
>
>
> Please, though - use {% else %}. Its totally clear what its referring
> to and else doesn't mean squat unless you see what the if (and in this
> case, for) test is anyway, so I don't think this would be confusing
> (after all, this isn't python).
> (Also, if you want to avoid confusion don't use a keyword that is
> located within another language's looping construct :)
>
>
> Please dont use else, because {%for%} matches python's for loop and 
> that supports an else clause which does not operate like this. If the 
> same keyword is used, it should behave the same.
>
>
> -- 
> Read my blog! I depend on your acceptance of my opinion! I am interesting!
> http://techblog.ironfroggy.com/
> Follow me if you're into that sort of thing: 
> http://www.twitter.com/ironfroggy
>
> >

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Pickling QuerySets

2008-09-27 Thread Ned Batchelder


The behavior of pickling QuerySets changed in qsrf, and therefore in 
Django 1.0: previously the pickle included only the query, now it 
includes the results.  This was the root cause of a bad problem that 
took us three days to find: 
http://nedbatchelder.com/blog/200809/a_server_memory_leak.html.  At one 
point the docs mentioned the pickling behavior explicitly, and now they 
don't.  Pickling isn't mentioned as one of the reasons a QuerySet is 
evaluated in the current docs 
(http://docs.djangoproject.com/en/dev/ref/models/querysets/), and 
there's no specific description of what happens when a QuerySet is pickled.

Am I missing where this is discussed, or is it an oversight, or is there 
a reason not to mention it?  I'll patch the docs if you'd like...

--Ned.

-- 
Ned Batchelder, http://nedbatchelder.com



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: How to add a new DB backend?

2008-07-16 Thread Ned Batchelder

I don't know much about this, but I suspect a huge issue will be the 
non-relational nature of CouchDB.  All of the existing backends execute 
SQL statements against their databases.

--Ned.
http://nedbatchelder.com

Malcolm Tredinnick wrote:
> On Wed, 2008-07-16 at 17:29 -0400, Michael Miller wrote:
>   
>> Hi,
>>
>> I'm interested in building a couchDB backend for django.  Is there  
>> any documentation on how to start adding a new backend?
>> 
>
> The code is the documentation here (it's Python, so it's just executable
> pseudo-code, after all). :-)
>
> If you're really able to replace it at the SQL level, pick one of the
> existing backends as a model and also read through
> django/db/backends/__init__.py. You'll need to implement the appropriate
> bits of BaseDatabaseOperations and BaseDatabaseFeatures in a subclass
> for the backend. Have a look at how, e.g, the SQLite backend does this.
>
> If you're needing to replace all of the query generation, you may need
> to replace the entire Query class from django/db/sql/query.py. In that
> case, look at how the Oracle backend creates its own Query class (it
> subclasses the standard one, but you might need to completely replace
> it).
>
> Almost all the tests are backend-agnostic. They're also backend-language
> agnostic for the most part (there are a couple of places we tests for
> the number of "left outer joins", but they're very limited). So the test
> suite will be very useful there.
>
> Regards,
> Malcolm
>
>
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django admin Error at /admin/auth/user/1/?

2008-03-31 Thread Ned Batchelder

Just to explain my motivation: it wasn't so much to optimize the core 
devs' time, though that is a concern, but to make on-ramping new users 
as friction-free as possible.  If there's a common error that newbies 
make, we can either provide an error message ("wrong group"), or try to 
fix the system so that they don't make the error in the first place.  
The latter is a better option where possible.  It's as true of the 
community as it is of the software.

--Ned.
http://nedbatchelder.com/blog

Malcolm Tredinnick wrote:
> On Mon, 2008-03-31 at 07:11 -0400, Ned Batchelder wrote:
>   
>> I figured the name of the group is hard to ignore.  By making the name
>> more specific, people would be more likely to find the appropriate
>> group.
>> 
>
> Yeah... I know where you're coming from. It's hard to tell if the change
> will work or not.
>
> Speaking only for myself, it's not that huge of a problem when people
> periodically post to the wrong list. We point them in the right
> direction and everybody moves on. If we all remain polite, nobody seems
> to take offence. As a percentage of traffic on this list, slightly
> misdirected posts are a small amount. So I tend to favour the "do
> nothing" action slightly at this stage, rather than chase an ephemeral
> perfect solution.
>
> Anyway, I've posted two more times to this thread that I probably needed
> to. It's not that big a deal for me either way. Also not my decision.
>
> Malcolm
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Public spec is needed for writing ORM adapters

2008-03-25 Thread Ned Batchelder

I thought I would end this standoff by simply going where James pointed, 
and turning it into the minimal "spec" Alex describes.  I know next to 
nothing about db backends, but it sounded like this was a job for a 
scribe, not a developer.

Unfortunately, django.db.backends.dummy falls below the bar for minimal 
API documentation.  While it may have the names of the entry points, 
they are all defined as complain(*args, **kwargs), and they have no 
docstrings or comments, so there is no guidance about what they expect, 
do, or produce.

Perhaps James mis-spoke: django.db.backends contains base classes, and 
has much more useful information.

--Ned.
http://nedbatchelder.com/blog

James Bennett wrote:
> On Mon, Mar 24, 2008 at 11:15 PM, Alex P <[EMAIL PROTECTED]> wrote:
>   
>>  1. The original message was in brief: there is a growing interest in
>>  the Python community for DB2 support, and we (developers behind IBM_DB
>>  driver, DB-API wrapper and SQLAlchemy adapter) are interested to help
>>  in the Django context if some _minimal_ documentation is provided in
>>  an unrestricted form, even under a Open Source license like BSD (even
>>  with sample/example API usage code, for that matter).
>> 
>
> The thing is, though, that there *is* a minimal API documented:
> django.db.backends.dummy. If this were Java, you could take that as
> laying out the interface to be implemented. The confusion came from
> the apparent refusal to deal with this because it's in the form of
> open-source code.
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Recent test breakage on Windows

2008-03-24 Thread Ned Batchelder

I was wary of re-implementing the logic in the test, just because it 
makes the test somewhat opaque.

In the end, I added a helper function to clean the path results, 
submitted a patch: http://code.djangoproject.com/ticket/6868

--Ned.
http://nedbatchelder.com/blog

Malcolm Tredinnick wrote:
> On Sun, 2008-03-23 at 11:15 -0400, Ned Batchelder wrote:
> [...]
>   
>> Here's one of the tests in question:
>>
>>  >>> f = forms.FilePathField(path=path)
>>  >>> f.choices.sort()
>>  >>> f.choices
>> [('.../django/newforms/__init__.py', '__init__.py'), 
>> ('.../django/newforms/__init__.pyc', '__init__.pyc'), 
>> ('.../django/newforms/fields.py', 'fields.py'), 
>> ('.../django/newforms/fields.pyc', 'fields.pyc'), 
>> ('.../django/newforms/forms.py', 'forms.py'), 
>> ('.../django/newforms/forms.pyc', 'forms.pyc'), 
>> ('.../django/newforms/models.py', 'models.py'), 
>> ('.../django/newforms/models.pyc', 'models.pyc'), 
>> ('.../django/newforms/util.py', 'util.py'), 
>> ('.../django/newforms/util.pyc', 'util.pyc'), 
>> ('.../django/newforms/widgets.py', 'widgets.py'), 
>> ('.../django/newforms/widgets.pyc', 'widgets.pyc')]
>>
>> On my machine, the problem boils down to:
>>
>> Expected: '.../django/newforms/__init__.py'
>> Got: 'C:\\src\\django\\django\\newforms/__init__.py'
>> 
> [...]
>   
>> Any guidance?
>> 
>
> Given that we can compute the expect result pretty easily (it's the
> directory listing of the that directory), I'd be tempted to compute the
> expected result as part of the test and compare it to what the widget
> puts out. This isn't quite as tight a coupling as it looks on the
> surface, since the interface definition is that it returns a list of
> tuples that are the files in that path. So we can write the "expected
> result" based on the interface description, not the implementation (yes,
> it's semantic hair-splitting, but it's type of trickery that helps me
> sleep at nights in those months when I sleep).
>
> I'm always a little wary of the "just add more dots" approach to ironing
> out these problems because unexpected test failures have caught a lot of
> problems over the past couple of years.
>
> Still, adding more leading dots wouldn't destroy the world order,
> either.
>
> Regards,
> Malcolm
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Recent test breakage on Windows

2008-03-23 Thread Ned Batchelder

I'm not familiar with PyUnit.  How would it be easier?

--Ned.

Nick wrote:
> On Mar 23, 3:37 pm, Ned Batchelder <[EMAIL PROTECTED]> wrote:
>   
>> That was my first instinct too, then I had to re-discover (for the
>> umpteenth time!) that normpath does not normalize a path to make it the
>> same across Python implementations: it normalizes the path to make it
>> right for the platform.  From the docs: "On Windows, it converts forward
>> slashes to backward slashes." D'oh, that just makes the problem worse...
>>
>> And it turns out changing ".../django/newforms" to "...newforms" in the
>> expected data isn't enough.  Where the files are two levels deep, there
>> are backslashes further in also:
>>
>> 
>
> Perhaps this can be fixed by changing the definition of the path
> variable, earlier in the test code.
>
> Then again, perhaps this is one instance where writing the
> tests using PyUnit would be easier than doing it with doctest.
>
>
> Nick
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: deprecated Model.init(*args)

2008-01-30 Thread Ned Batchelder


I would have thought Adrian's deferred fields proposal would fall 
squarely in the post-1.0 bucket.  It clearly has no backward 
compatibility issues, so it can be added after 1.0.  Maybe you aren't 
proposing to include it in 1.0; it wasn't clear from your message.

I'm not arguing against removing positional argument support from model 
constructors, just wondering about the 1.0 focus.

--Ned.

Jacob Kaplan-Moss wrote:
> Howdy folks --
>
> The short version:
>
> I'd like to deprecate initializing models using positional arguments
> (i.e. ``p = Person(1, 'Joe Somebody')``) in favor of only allowing
> keyword-argument initialization (i.e. ``p = Person(id=1, name='Joe
> Somebody')``).
>
> I'd make the ``*args`` style start issuing DeprecationWarnings
> immediately, and remove support entirely when we wipe deprecated
> features in the run-up to 1.0. I'd make this change on the
> queryset-refactor branch.
>
> Long version, with explanation:
>
> This week I've been starting to help Malcolm out the queryset-refactor
> branch. To get my feet wet I've been playing with Adrian's deferred
> fields proposal (http://code.djangoproject.com/ticket/5420).
>
> Turns out it's trickier that I'd though, but mostly because of the
> fact that QuerySets initialize models using ``Model.__init__(*args)``
> instead of using kwargs. I'm almost certainly going to have to change
> the internal behavior to get deferred fields working OK.
>
> As I started down that road, though, I realized that positional
> initialization of models is something I've only seen done internally
> to Django. Further, using this "feature" can lead to all sorts of
> nasty bugs: if you change the order of fields in models.py all of a
> sudden fields start getting the "wrong" values from positional
> initialization. On top of that, removing ``*args`` support from
> ``Model.__init__`` would make the code cleaner and a bit faster.
>
> Thoughts?
>
> Jacob
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: How to stop spam on our groups

2008-01-12 Thread Ned Batchelder

The moderation could be done by member(s) whose time wouldn't be spent 
enhancing django.  I used to volunteer on python.org to maintain the 
Python Jobs Board, because I wanted to help out, but I wasn't able to 
contribute in ways that other people already were (sysadmin stuff).  
Perhaps if there were two or three people who cared about Django but 
were not otherwise engaged in the improvement of Django, they could help 
by taking the moderation and/or cleanup task off the shoulders of the 
maintainers.

The problem with this scheme is finding people who will do it reliably, 
and won't suddenly drop off without warning.

--Ned.

Simon Greenhill wrote:
> The major problem is that someone has to moderate. Who does this?
> Moderating would take a non-trivial amount of time - especially on the
> high traffic django-users. Wouldn't we rather have djangonauts
> enhancing django than cleaning up spam?
>
> I suspect that most people following the lists are getting the
> messages sent to their email accounts (instead of reading via web-
> interface), and then their spam filters are probably killing most of
> these.
>
> -Simon
>
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Django 1.0 features -- the definitive list

2007-12-04 Thread Ned Batchelder

I would like to see a replacement for the regex scheme as well, but not 
because I am uncomfortable with regexes.  I think the typical regex for 
a URL is noisy: it's hard to see the intent of the expression.  Most URL 
regexes follow some very well-defined patterns, and we don't have a way 
to express them other than to "compile them down" to regexes.  Sometimes 
you need the full power, but usually, your URL consists of fixed parts 
and variable parts separated by slashes.  The variable parts are often 
numbers, or slugs, and need to be given names.  Being able to express 
that succinctly would solve most people's URL matching.

I don't have a suggestion for a replacement, and I don't think it needs 
to be on the 1.0 list (since it can be added without breaking backward 
compatibility), but I think it would be a big plus.

--Ned.
http://nedbatchelder.com/blog

Simon Willison wrote:
> On 4 Dec 2007, at 13:26, bobj wrote:
>
>   
>> Simon - These are GREAT!!! Ideas. The regular expression based URL
>> dispatching  replacement has been something I personally have been
>> thinking about for some time.  I would be interested in helping with
>> this If you put together a proposal. One URL implementation worth
>> considering is "Routes" ( http://routes.groovie.org ). I have used
>> Routes for web applications and I think it is easy to grok. Have you
>> taken a look at Routes?
>> 
>
> I like Routes in principle, but the way it has the concept of a  
> "controller" baked in to it didn't sit very well with me. I can't  
> imagine it would be hard to use it without controllers though.
>
> I'm generally pretty happy with regular expressions, but I watched  
> Scott Guthrie's presentation on the ASP.NET MVC framework a few weeks  
> ago (it's really interesting, they've taken a bunch of ideas from  
> Django and it gets name-checked in the presentation) and one of the  
> things he noted is that there are developers (especially in the  
> Microsoft ecosystem) who never really got comfortable with regexps. I  
> don't want those people to be turned away from Django on the first  
> page of the tutorial. He also described the ASP.NET MVC syntax for  
> URLs, which looks like this:
>
> search/[query]
> product/[id]
>
> http://weblogs.asp.net/scottgu/archive/2007/12/03/asp-net-mvc- 
> framework-part-2-url-routing.aspx
>
> While I personally prefer the power of regexps, I can see how the  
> above would make it much easier for developers just beginning to get  
> their feet wet with Django. We can always enlighten them with the  
> power of regular expressions once they're hooked. If URL handling is  
> interchangeable we can have the best of both worlds. Incidentally,  
> allowing URL logic to be swapped out was an initial design goal back  
> when we put the URL stuff together - we were worried about  
> performance, but when we benchmarked it turned out that Python can  
> run through a list of a hundred or so regular expressions in less  
> than 0.01 seconds.
>
> Cheers,
>
> Simon
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Still no favicon - Re: Visual recognition of Django website

2007-11-02 Thread Ned Batchelder


I don't know who did it or when, but the http://djangoproject.com site 
is now proudly displaying a favicon.  Thanks to whoever it was for the 
Halloween treat...

--Ned.

Mikkel Høgh wrote:
> If no one is against this, why hasn't anything happened yet?
> If Jacob, or anyone else, is against it, I wish they would step
> forward and say so. Perhaps even argue as to why.
>
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Discussion of #4131: addslashes, javascript string literals, and (new) escapejs

2007-10-19 Thread Ned Batchelder

I liked the original proposal (mine!) to extend addslashes rather than 
to nearly-duplicate its functionality in another filter, but I can see 
the logic of slicing these things finely.  I like that the docs for 
addslashes refers to escapejs.  For the purposes of education, the 
escapejs docs should explain why they convert  On 10/17/07, Jeremy Dunck <[EMAIL PROTECTED]> wrote:
>   
>>   However, there are many strings that can be passed through that
>> filter and sill will break javascript string literals.
>> 
>
> Specifically, as you point out, strings that contain "" --
> the main point here is to reduce the chances of XSS attacks when
> embedding user-originated data into scripts.
>
>   
>>   4131 now has a patch (from Andy Durdin) which would introduce a new
>> defaultfilter named escapejs.  It does the complete job of escaping
>> anything that could break out of a string literal.
>> 
>
> Credit where it's due;  The meat of the patch is Jeremy's, I just
> tidied it up a tad.
>
> Andrew
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Visual recognition of Django website

2007-09-19 Thread Ned Batchelder

I agree that a favicon is one of those fit-and-finish touches that helps 
complete a website.  Attached are my attempts.  I agree with Todd that 
"dj" is a better reminder of Django, and the color should be greener 
(look at the badges to see that Wilson didn't slavishly follow the deep 
green of the logos when making smaller versions of it). I've also 
rejiggered the letters a bit to avoid smeary-looking blurs.

Now we just need to get someone to put it on the site...

--Ned.



Todd O'Bryan wrote:
> On Wed, 2007-09-19 at 00:29 +, Johann C. Rocholl wrote:
>   
>> On Sep 17, 10:17 am, "Justin Lilly" <[EMAIL PROTECTED]> wrote:
>> 
>>> I personally would also like a favicon for the django sites. I took the
>>> liberty of creating one using django's colors and fonts (stole the d from
>>> the logo).
>>>
>>>  http://www.flickr.com/photos/[EMAIL PROTECTED]/1397125183/
>>>   
>> Here's another attempt, with improved vertical alignment
>> (mathematically perfect), and in Windows ICO format:
>> http://johann.rocholl.net/django-icon/favicon.ico
>>
>> 
>
> While I'm +0 on the favicon thing, I personally think "d" is
> insufficient and "dj" would be much more identifiable as uniquely
> Django.
>
> Todd
>
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

<><>

Re: Cleaning up memcached connections

2007-08-10 Thread Ned Batchelder

This rang some bells about a problem we'd had with memcached, so I asked 
Dave St. Germain (our goto guy for issues like this).  Here's what he 
had to say:

I think this is the wrong fix for the problem.  memcached
connections are supposed to be persistent, not torn down after every
request.  The client is working exactly as it's supposed to.  They
could certainly run out of connections to the memcached server if,
for instance, they had apache configured in worker mpm mode, which
would mean that there'd be a separate connection per thread (and by
default, there are I think 25 threads per apache worker process
[times at least 2 processes]).  So, yeah, they could quickly create
a bunch of idle connections, but I wouldn't consider it a "bug" in
memcached or django.
The patch that I contributed actually "caused" this problem in the
apache worker scenario.  Previously, one connection was shared per
process (no matter how many threads), but that lead to much worse
problems as the memcache client wasn't threadsafe at all.
Instead of closing the connection every request, they could make the
memcache client synchronized (or the django wrapper to the client),
though that doesn't seem like a great idea, either.

--Ned.

Jacob Kaplan-Moss wrote:
> Hi all --
>
> We've noticed that in a few cases Django under mod_python can leave
> dangling connections to memcached open. We've had trouble tracking
> down the circumstances under which this happens, but when it does it
> can lead to memcached servers hitting their connection limits, which
> means caching stops. Nasty.
>
> I've fixed the problem, and the patch is at
> http://code.djangoproject.com/attachment/ticket/5133/memcached-cleanup-connections.patch,
> but I'm not sure about the ramifications of the fix. Thoughts before I
> check this in?
>
> Jacob
>
> >
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: None != Null?

2007-07-17 Thread Ned Batchelder

>> but there always are). Remember that we're writing the ORM for Python
>> coders, so using natural-looking Python behaviour is important.
>> 
>
> Well, there are two ways to interprete this particular lookup. 
>
> The ORM is an ORM for relational database users, and you won't get far if
> you have no idea what NULL is. Django does not abstract away the relational
> database. If you don't know what a relational database is, you probably
> better off using Zope ...
>
> I consider your point of view as valid as mine. And it was brought up in the
> original discussion, too.
>
>   
>>> I guess you cannot make both groups completely happy, but your proposed
>>> change is also backwards incompatible and has been explicitly decided
>>> different before. Can you please reconsider?
>>>   
>> I'm not convinced about the huge negative impact of the change, either.
>> There was *one* use-case touted (by Deryck Hodge) in the original
>> thread. It has limited applicability. It's only for disjunctive ("or")
>> filters, since a sequence of conjunctions like
>> filter(foo=None).filter(bar=6) is just a complicated way of writing the
>> empty list unless you're using, say, PostgreSQL 7.1 or something else
>> that doesn't handle NULLs correctly.
>>
>> I didn't like this change when it went in and I wouldn't be unhappy to
>> see it fixed. As usual, though, it's not the complete end of the world
>> if it doesn't change, just makes it a less beautiful place.
>> 
>
> Russell has based the decision on uses within the django tests that appear
> to come naturally for him, too, so I don't share your opinion that this
> change wouldn't affect anything. Also, it questions your opinion that the
> current semantics are pointless:
>
> http://groups.google.com/group/django-developers/msg/b3f7e4cf6f9ffd98,
> Russell Keith-Magee wrote:
>
> --- quoting Russell ---
> Hence the documentation to that effect. No, it isn't a particularly 
>  useful expression, but it does have at least one use case (discussed 
>  below). 
>  
> I wasn't aware of the Oracle issue; but that strikes me as a problem 
>  that should be fixed at the backend level, rather than as a reason to 
>  exclude a feature. 
>
> [On 10/14/06, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote:]
>   
>> So doesn't this defeat the purpose of the exercise now? Using 
>> 
>  > __exact=None is now a synonym for "please don't ever return anything 
>  > (except on Oracle)" whereas I thought your were shooting for "where this 
>  > field is NULL". 
>  
> The use case that started this for me is in the regression test - a 
>  related manager for an unsaved object. In this case, 'don't ever 
>  return anything' is _exactly_ the right response, because nothing can 
>  be related to the unsaved object. 
>  
> Mapping __exact=None to _isnull=True will give effectively the same 
>  results, but via a 'return everything with a primary key value of NULL 
>  - which is nothing' interpretation of the problem. 
>  
> However, as you noted earlier, this second approach is redundant given 
>  the existence of _isnull. It also hides a path to the expression of 
>  some valid (albiet edge case) SQL. The interpretation suggested by 
>  Michael struck me as a good way to avoid the redundancy, fix the bug I 
>  found, expose a small corner of legal SQL statements, and ultimately 
>  required less code to implement. 
>  
> The biggest problem I could see was the potential for '__exact=None 
>  doesn't mean what you think it does' amongst newcomers. For me, the 
>  best solution to this problem is documentation. 
> --- end of quote ---
>
>
> So (really) long,
>
> Michael
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Ned Batchelder's hyphenate

2007-07-10 Thread Ned Batchelder

Since the algorithm is identical to the one used by TeX, the hyphenation 
data can be taken from there as well.  I used a TeX distribution to get 
the latest patterns for English to include in the module.  I installed 
MiKTeX, and dug around in the tex/generic/hyphen directory to find 
them.  There are also French and German patterns in that distro, and 
there may be other hyphenation data sets in other repositories on the 
web, I haven't looked.

--Ned.

[EMAIL PROTECTED] wrote:
> On further reflection, there is a huge internationalization issue
> here. The hyphenation rules and data driven exceptions are English
> specific. Some will work (minimally) for other languages, but are not
> good enough. Proper integration will be required, and language
> developers will need to have more knowledge about this corner domain.
> Due to my NDA/NC I cannot work on that part of it, but I do have a
> patch almost ready for django.utils.text.wordwrap to take an optional
> boolean argument to do word hyphenation.
>
> Thankfully it is data driven and getting the data from the .po should
> not be too difficult. The problem will be getting the initial data.
>
> On Jul 9, 10:30 pm, "[EMAIL PROTECTED]"
> <[EMAIL PROTECTED]> wrote:
>   
>> Ned just posted the code for the tabblo hyphenate filter in the public
>> domain. This should be added as a builtin django filter with proper
>> attribution. I don't think wordwrap should use it by default, and
>> optional arguments don't work. I was thinking of just calling it
>> 'hyphenate' or 'hyphenatedwordwrap'.
>>
>> http://www.nedbatchelder.com/code/modules/hyphenate.html
>>
>> Thoughts?
>> 
>
>
> >
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Ned Batchelder's hyphenate

2007-07-10 Thread Ned Batchelder

Todd, good to meet a fellow nerd: I also have the five-volume hardcover 
set. My code is implemented from appendix H of volume 1 (or is it volume 
A?).

--Ned.

Todd O'Bryan wrote:
> On Tue, 2007-07-10 at 02:54 +, [EMAIL PROTECTED] wrote:
>   
>> On further reflection, there is a huge internationalization issue
>> here. The hyphenation rules and data driven exceptions are English
>> specific. Some will work (minimally) for other languages, but are not
>> good enough. Proper integration will be required, and language
>> developers will need to have more knowledge about this corner domain.
>> Due to my NDA/NC I cannot work on that part of it, but I do have a
>> patch almost ready for django.utils.text.wordwrap to take an optional
>> boolean argument to do word hyphenation.
>> 
>
> I seem to remember that Knuth did a pretty amazing job with hyphenation
> in TeX, most of it was algorithmic, and there were hyphenation engines
> for at least a few languages.
>
> I'll have to dig out my copy of The TeXbook to look (yes, I'm one of
> those nerds who has the five-volume box set of TeX and MetaFont books),
> but this may be something that somebody's already done a really good job
> on.
>
> Todd
>
>
> >
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Proposal: Let session support backends

2007-06-06 Thread Ned Batchelder

It sounds great, but memcached is a cache rather than a persistent 
store, so it doesn't guarantee to be able to give you back the data.  Is 
it OK for a session to drop because memcached flushed it out?  Or are 
you assuming you can size your memcached servers so that they never drop 
a session?

--Ned.

Jacob Kaplan-Moss wrote:
> On 6/6/07, Faulkner <[EMAIL PROTECTED]> wrote:
>   
>> Anyone seriously considering making a framework for this?
>> 
>
> I, for one, would be all for a (drop-in, API-compatible) session layer
> replacement with pluggable backends. Memcached sessions are a Good
> Idea.
>
> Jacob
>
> >
>
>
>
> .
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Escaping in templates...

2007-04-17 Thread Ned Batchelder

I've been following this discussion with interest.  XSS fragility is a 
real weak point for text-based templating engines, and we need to find a 
solution.

On the topic of HTML-escaping vs. general escaping: Absolutely the 
reason to do auto-escaping is to make it dead easy to avoid XSS 
problems, and so HTML escaping is easily the most important thing to get 
right.  While Django's dedication to template agnosticism is great 
(allowing emails to be generated with templates, for example), by far 
most of the text generated through the template engine is HTML, and that 
is the most vulnerable part of the Django ecosystem.

That said, though, keep in mind that not all text in a .html template is 
HTML:

My first variable is {{my_var1}}

var my_second_variable = "{{my_var2}}";
blah();

In this case, my_var1 needs to have "escape" applied.  The case of 
my_var2 is a bit trickier.  "addslashes" is good, but isn't enough 
(since  appearing in my_var2 will cause problems).  Things can 
of course get even trickier:

document.write("<p>{{my_var3}}</p>");

My brain starts to hurt trying to figure out how to protect my_var3!

--Ned.

Simon G. wrote:
> This is one of those issues which is never going to please everyone.
>
> So - I've started a list of the various proposals (1), and could you
> all add any other proposals to this page, along with any pros/cons,
> and vote on the one(s) you prefer.
>
> This way we can get some idea of what a consensus view might look like
>
> --Simon
> [1] http://code.djangoproject.com/wiki/AutoEscapingProposals
>
>
> >
>
>
>
> .
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #3527 - better debug traceback with code executing...

2007-04-09 Thread Ned Batchelder

If the only concern here is that debugging is a powerful feature that 
needs to be carefully controlled, then surely a setting to enable it is 
the right way to go?  After all, many security experts will tell you 
that the traceback handler we have now is a security hole, not because 
it lets you execute code, but because it reveals inner workings of the 
server, which can expose vulnerabilities.

Production servers need to have their settings carefully set.  That is 
true now, and it will be true if we add a DEBUGGER=True setting to 
enable this more powerful feature.  I say we add the power and let the 
administrators control where it appears.

--Ned.

Malcolm Tredinnick wrote:
> Hi,
>
> On Mon, 2007-04-09 at 04:33 -0700, jedie wrote:
>   
>> Why has django not a interactive AJAX traceback debugger?
>>
>> Using a interactive debugger you can play with objects at any point in
>> the error traceback.
>>
>> I known, a web shell is a open security hole. But the interactive
>> debugger should only running with the development Web server and if
>> debugging is on.
>> The development server is not for production use. So there is IMHO no
>> problem.
>> 
> [...]
>   
>> Existing django ticket: http://code.djangoproject.com/ticket/3527
>> 
>
> The reason I originally closed that ticket as "wontfix" -- and I still
> think it's the right reasoning -- is because the debug traceback handler
> is not associated with whether or not the development server is running.
>
> Instead, it is triggered by whether or not DEBUG is True. Sometimes you
> want to have DEBUG=True in production environments, whether for just a
> little period of time -- to debug something -- or for longer. So I am
> reluctant to put in something that might be a security hole if there's
> any chance of it being run on a production site.
>
> Regards,
> Malcolm
>
>
>
> >
>
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Stack trace in DB comments

2007-03-30 Thread Ned Batchelder

The code is not online (yet).  I plan to, but will need to wait for the 
HP acquisition to settle down first...

--Ned.

Jeremy Dunck wrote:
> On 3/30/07, Ned Batchelder <[EMAIL PROTECTED]> wrote:
>   
>>  It wasn't a formal presentation.  I was showing Tabblo during the Django
>> app demo session, and Adrian noticed that my dev server was spewing more
>> info than typical.
>> 
>
> Yeah, I figured it was the app demo.  I was asking if the middleware
> was online, not presentation slides.  :)
>
> >
>
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Stack trace in DB comments

2007-03-30 Thread Ned Batchelder

It wasn't a formal presentation.  I was showing Tabblo during the Django 
app demo session, and Adrian noticed that my dev server was spewing more 
info than typical.

--Ned.

Jeremy Dunck wrote:
> On 3/30/07, Ned Batchelder <[EMAIL PROTECTED]> wrote:
>   
>> At Pycon, when I showed
>> my tracing middleware, I got the distinct feeling that its seven new
>> settings were a no-no.
>> 
>
> I guess I missed that pres.  Is it available online somewhere?
>
> >
>
>
> .
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Stack trace in DB comments

2007-03-30 Thread Ned Batchelder


I think it would be a great idea.  I have used similar tracing 
facilities in other (non-Django) projects, and they were invaluable for 
quickly zeroing in on the pathological code that swamped the performance.

I don't know what the feeling of the devs is on cluttering the settings 
with individual switches for things like this.  At Pycon, when I showed 
my tracing middleware, I got the distinct feeling that its seven new 
settings were a no-no.  For example, the settings are a flat namespace, 
with no provision for partitioning.

--Ned.

Jeremy Dunck wrote:
> In the book "Building Scalable Web Sites", Cal Henderson suggests
> putting (possibly abbreviated) app call stacks  in SQL comments to
> enable logging and performance on the DB backend to be more easily
> tied to application paths.
>
> This sounds like a great idea to me.
>
> MySQL, PGSQL and SQLite all support the same in-query comment syntax.
>
> I can see two ways to implement this.  One would be an augmentation to
> utils.CursorDebugWrapper.  The other would be a new wrapper.
>
> Doing it in CursorDebugWrapper seems a fairly natural place, but using
> that (easily) requires settings.DEBUG to be true.  You probably
> wouldn't want to do that in production.
>
> Doing it in a new wrapper, we could introduce settings.DB_DEBUG which
> would -only- to the trace comments in the SQL, and would therefore be
> safe to do in production.  (We could still make it such that if
> DEBUG=True, DB_DEBUG=True.)
>
> Either way, I think this would be a simple and useful change.
>
> Thoughts?
>
> >
>
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Add a salt to the newforms fields names

2007-03-23 Thread Ned Batchelder


I tried hacking around with newforms, to implement part of my Stopping 
Spambots with Hashes and Honeypots 
(http://www.nedbatchelder.com/text/stopbots.html).  My approach was to 
create a BotProofForm class which would wrap an instance of an ordinary 
form.  This let me rename fields without changing the underlying 
implementation of BaseForm (or BoundField).

I didn't get everything working, but I think the approach would work for 
hashing field names.  More difficult was how to add honeypot fields, 
since hiding them requires changing the HTML representation of fields.

--Ned.

Baptiste wrote:
> Hello all,
>
> (Please apologize my bad English, don't mind about it and try to
> understand... I'll do my best !)
>
> First, have a quick look on the spammer main method - I am talking
> about bots, not human spammers that can't be filtered - : POST data
> are sent to the server with classical names of fields, like "comment",
> "website", "title", ...
> One of the method to prevent or limit spam is to display altered field
> names that the bots can't identify.
>
> I propose me to work on that... but first, I wanted to know if it was
> an interesting feature, and if we could chose a protocol to make these
> things work. So these are some ideas :
> * The BaseForm constructor has a new parameter : "scramble". Set to
> True (default False), the things could work like that :
> in _html_output(), for each field, we generate a md5 hash of the name
> +a secret+a date(no day, just week, month, year, because an user can
> keep a page open a few days, not a week. that will prevent the
> recording of the hash, if date is in it). We pass this string to each
> BoundField, and all works like before. In the generated code, names
> don't mean anything and cannot be used by spammers to identify fields.
> * If the user gets back data returned by a scrambled form : he
> instantiates a form with these data, the parameter scramble to True,
> and the form will have a different behaviour. It will treat the data
> by doing a loop on the dictionary, recreating the hash, and comparing
> it with fields names to make a new dictionary with correct names and
> values, what would allow the user to do again form['name'].
> It may sound a little bit confuse - it is. So please tell me what you
> think about that, and how you would do it.
>
> Regards
> Baptiste.
>
>
> >
>
>
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Critical ticket: can cause data loss

2007-01-29 Thread Ned Batchelder

I think it is risky to assume that all potentially affected users will 
see this information in this thread.  I found it, but there was an 
element of luck involved.  I don't read this list religiously, and could 
easily have missed it. 

I don't know that you need to make a release for this fix, but I think 
you should alert people to it.  There is a mailing list for announcing 
releases and security issues, right?  You should send a message to that 
list announcing the patch.

--Ned.

Adrian Holovaty wrote:
> On 1/25/07, James Bennett <[EMAIL PROTECTED]> wrote:
>   
>> At the moment I'm leaning toward rolling 0.95.2 immediately after that
>> goes in, but I'd like to hear opinions on it; with 0.96 probably
>> coming up soon, I can come up with good arguments either way.
>> 
>
> Nah, given that generic relations have never been documented and the
> small fraction of people using them probably all subscribe to this
> mailing list, I don't see any point in having another release. Plus,
> 0.96 is coming soon enough.
>
> Adrian
>
>   

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Error message when using wrong filter keyword could be more helpful

2006-10-29 Thread Ned Batchelder





Yes, sorry about the unit tests.  I'll look into that.

As for the enhancement request, we'll have to consider this email
thread the request, at least until Akismet settles down.

--Ned.

Russell Keith-Magee wrote:

  On 10/29/06, Blake Winton <[EMAIL PROTECTED]> wrote:
  
  

  Much nicer would be to show the list of field name that can be used:
TypeError: Cannot resolve keyword 'storyitem' into field, choices are:
tags, event, admintag, storycomment, favorite, story, storyundo,
  

  
  ...
  
  
Even nicer would be to sort the list of field names before printing them...

  
  
And nicer still would be a patch that fixes all the unit tests that
break when this patch is applied :-)

I like the idea. However, Malcolm is currently in the process of
refactoring the query code, so we're trying to keep changes in this
area to a minimum. I would suggest logging this as an enhancement
request so that it doesn't get forgotten.

Yours,
Russ Magee %-)





  


-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Error message when using wrong filter keyword could be more helpful

2006-10-27 Thread Ned Batchelder

(This is a ticket that Akismet wouldn't allow.)

When I use the wrong filter keyword (for example, a typo in a field 
name), the error message says,

TypeError: Cannot resolve keyword 'storyitems' into field

Much nicer would be to show the list of field name that can be used:

TypeError: Cannot resolve keyword 'storyitem' into field, choices are: 
tags, event, admintag, storycomment, favorite, story, storyundo, 
feature, textblock, textblock, shareablelink, groupstory, textblock, 
storyace, invitation, storyview, page, postcard, poster, id, name, date, 
modified_date, template, template2, access, status, user, 
caption_policy, share_policy, aspect, layout, flowmode, collmode, 
widget, widgetstale, revision, undostate, karma, storydesign, type, 
label, parent, product_id

This helps me get my code working faster, with less head-scratching.

Attached is the patch to query.py that enables these better messages.

--Ned.

-- 
Ned Batchelder, http://nedbatchelder.com



--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
--- django/db/models/query.py   (revision 3943)
+++ django/db/models/query.py   (working copy)
@@ -752,12 +752,19 @@
 else:
 matches = [f for f in field_list if f.name == name]
 if len(matches) != 1:
 return None
 return matches[0]
 
+def field_choices(field_list, related_query):
+if related_query:
+choices = [f.field.related_query_name() for f in field_list]
+else:
+choices = [f.name for f in field_list]
+return choices
+
 def lookup_inner(path, lookup_type, value, opts, table, column):
 qn = backend.quote_name
 joins, where, params = SortedDict(), [], []
 current_opts = opts
 current_table = table
 current_column = column
@@ -830,13 +837,17 @@
 
 raise FieldFound
 
 except FieldFound: # Match found, loop has been shortcut.
 pass
 else: # No match found.
-raise TypeError, "Cannot resolve keyword '%s' into field" % name
+choices = field_choices(current_opts.many_to_many, False) + \
+
field_choices(current_opts.get_all_related_many_to_many_objects(), True) + \
+field_choices(current_opts.get_all_related_objects(), 
True) + \
+field_choices(current_opts.fields, False)
+raise TypeError, "Cannot resolve keyword '%s' into field, choices are: 
%s" % (name, ", ".join(choices))
 
 # Check whether an intermediate join is required between current_table
 # and new_table.
 if intermediate_table:
 joins[qn(current_table)] = (
 qn(intermediate_table), "LEFT OUTER JOIN",

Re: bump: give Gabor (or somebody else) a unicode branch in svn

2006-10-15 Thread Ned Batchelder





I can understand the desire to keep the branch count low, but looking
at the number of people asking for Unicode support, and the broad
applicability it would have, and the comparitvely low demand for
row-level permissions and multi-db support, I would argue for starting
the Unicode branch now.  It may be finished, tested, merged and on the
trunk before the other branches are done.  I know that will make
merging the other branches more difficult, but Unicode support is in
high demand.

--Ned.

Adrian Holovaty wrote:

  On 10/13/06, Ivan Sagalaev <[EMAIL PROTECTED]> wrote:
  
  
Max Derkachev wrote:


  There were a lot of words about unicodification of Django, but things
has not been moved a bit further.
  

Or at least it would be nice to hear from someone of the core team why
it can't be done (either right now or at all). I remember Jacob (I
think) has said once along the lines that he's not very excited about
unicodification since because of too little gain from too much effort.
But the branch will be maintained by other people so why not try?

  
  
Hey there,

I'm very interested in getting this done, but we've got quite a few
branches open at the moment. I think we should focus on merging at
least one of these branches before opening another one, for the sake
of everybody's sanity.

Adrian

  


-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: Contrib context processor?

2006-10-13 Thread Ned Batchelder





Only a security hole in the sense that a template author could put the
DB password onto the page (for example) if they were stupid or
malicious, right?  I understand the desire to "protect" template
authors from the full power of Python etc, but we don't believe they
are untrusted, or do we?

--Ned.

James Bennett wrote:

  On 10/13/06, Ned Batchelder <[EMAIL PROTECTED]> wrote:
  
  
 In my own context processor, I added 'settings' as the entire settings
module.  Then I can get settings.WHATEVER in the templates.  This solved our
problem of dribbling in individual settings as we needed them.  Any feelings
about doing that in a standard context?  Then there is no slippery slope, as
all of the settings are brought in at once.

  
  
That opens up some potential security holes which I'm not certain
could be worked around by just listing the "safe" settings.

  


-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: Contrib context processor?

2006-10-13 Thread Ned Batchelder





In my own context processor, I added 'settings' as the entire settings
module.  Then I can get settings.WHATEVER in the templates.  This
solved our problem of dribbling in individual settings as we needed
them.  Any feelings about doing that in a standard context?  Then there
is no slippery slope, as all of the settings are brought in at once.

--Ned.

James Bennett wrote:

  On 10/13/06, Jeremy Dunck <[EMAIL PROTECTED]> wrote:
  
  
I'm adding it locally, but was wondering if you'd like a patch for one
either to core or contrib?

  
  
It came up in ticket 2532, and Adrian vetoed it on the grounds that
adding context processors for some settings could be a slippery slope
toward having a context processor for every setting.



  


-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: Find the Cookie owner

2006-10-12 Thread Ned Batchelder





You'll probably find it much easier to use Django to serve those static
files, rather than trying to duplicate Django functionality in
modpython.

--Ned.

Mario Gonzalez ( mario__ ) wrote:

   Hi, I'm writing a code for a media server and I want to serve static
files to authenticated users only. I check against Django's session
table (django_session) and that's ok (IMO) but in session_data there
isn't the userid and I need it for security reasons; So I sent you
what I'm doing so far and please, I'd really like that someone can
help me a bit if you please.

  Many thanks!


PS: Greetings from Chile.
  
  

from mod_python import apache, Cookie
from os import environ

def accesshandler(req, **kwargs):
"""
(Was) Authentication handler that checks against Django's auth database.
(Is)  Access handler that check agains Django's session table
"""

options = req.get_options()
settings_module = options.get('DJANGO_SETTINGS_MODULE', None)
if settings_module:
environ['DJANGO_SETTINGS_MODULE'] = settings_module
else:
return apache.HTTP_FORBIDDEN

cookies = Cookie.get_cookies(req)

if cookies.has_key('sessionid'):
django_sessionid = cookies['sessionid'].value
else:
return apache.HTTP_FORBIDDEN

from django import db
db.reset_queries()

cursor = db.connection.cursor()
sql = """
  SELECT session_data
  FROM django_session
  WHERE expire_date > now()
   AND session_key = '%s'
""" % django_sessionid
cursor.execute( sql )
session = cursor.dictfetchone()

sessionid_is_found = False
if len(session['session_data']) > 0:
sessionid_is_found = True

if not sessionid_is_found:
return apache.HTTP_FORBIDDEN

import base64
a = base64.decodestring( session['session_data'] )

#who is the owner of this cookie??!
#cause in session['session_data'], is not
    req.write(a)


return apache.HTTP_UNAUTHORIZED

  


-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: single-row database tables for properties

2006-09-21 Thread Ned Batchelder


[EMAIL PROTECTED] wrote:
> No you dont need a different table for each property, one table stores
> all the properties that you want to collect together.
>
> The problem with the (name,value) approach is that all the values have
> to be the same type, so you cant present some properties as text, some
> as boolean, and some as drop-down choices. Also you might want to
> constrain the set of allowed names, and not allow additions.
>   
I see now.  You are adding new columns for the different properties, 
right?  There's no example in your original posting.

You are right: the name, value method does restrict you to only one 
value type.

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: Custom default managers interfere with delete operations

2006-09-11 Thread Ned Batchelder





Done: http://code.djangoproject.com/ticket/2698

--Ned.

Russell Keith-Magee wrote:

  On 9/10/06, Ned Batchelder <[EMAIL PROTECTED]> wrote:
  
  
No filtering allowed.

  
  
Hrm. I think you might be right. Taking a closer look, it seems that
the deletion algorithm is fairly conservative; I can't see a way of
constructing a non-default related manager that would exclude an
object from deletion without breaking referential integrity.

Lodge an ticket; try your hand at a patch if you're feeling adventurous.
  

-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: Custom default managers interfere with delete operations

2006-09-09 Thread Ned Batchelder







Russell Keith-Magee wrote:

  
  On 9/9/06, Ned
Batchelder <[EMAIL PROTECTED]
  > wrote:
  



   

I'm not familiar with the internals of this code, but here's a naive
conception: since the default objects manager is made automatically by
the ORM, couldn't it still be available internally for use in this
case?  Whatever filtering is done by the app developer's default
manager, it has to be defeated in order to properly delete related
objects.  So why not skip the developer's default manager and use the
default default manager (if you know what I mean).

  
  
This approach sounds possible. 
  
My lingering concern is that there might be situations where using the
user defined manager would be the correct behaviour for the object
deletion algorithm. Your example is a bit of a weird case, born of the
collision of two methods of 'deleting' objects (your boolean field
representation and Django's database level deletion). I'm not entirely
convinced that fixing the problem for your (somewhat specialised) use
case wouldn't break it for all other legitimate uses.
  
  
  

The details of my specific case are a bit of a red herring.  The fact
remains that if the default manager does any filtering at all, then
deleting an object may fail because its related objects will not all be
deleted, and the database's referential integrity will prevent the
deletion.  When looking for related obejcts to delete, you have to
always get all of them.  No filtering allowed.
-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: Custom default managers interfere with delete operations

2006-09-08 Thread Ned Batchelder

Russell Keith-Magee wrote:

On 9/8/06, Ned
Batchelder <[EMAIL PROTECTED]>
wrote:

The docs for custom managers say: "it's generally a good idea for the
first Manager to be relatively unfiltered". But it seems that any
filtering in the default manager will interfere with cascading deletes.

Having a filtered manager will cause some difficulty with cascading
deletes. The algorithm that finds the objects to be deleted uses the
related descriptors to traverse related objects; the related
descriptors are created based upon the default manager. If I say
myarticle.authors, Django uses the Author default manager to process
queries.

So, if your default manager for a given class excludes 'deleted'
objects, 'deleted' objects will not be discovered as part of the
related object set.

Is this the 'right' behaviour for descriptors (and by extension, the
deletion algorithm)? IMHO, it's an appropriate default. However, given
that you can define alternate managers for a class, I will conceed that
it is a little strange (and occasionally awkward) that you can't access
those managers through related descriptors.

I can't say I have a nifty solution to this problem, though. As always,
suggestions are welcome.

Yours,
Russ Magee %-)

I'm not familiar with the internals of this code, but here's a naive
conception: since the default objects manager is made automatically by
the ORM, couldn't it still be available internally for use in this
case? Whatever filtering is done by the app developer's default
manager, it has to be defeated in order to properly delete related
objects. So why not skip the developer's default manager and use the
default default manager (if you know what I mean).

--Ned.
--
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---

Custom default managers interfere with delete operations

2006-09-08 Thread Ned Batchelder


I don't know whether this is a bug in the ORM, or a misunderstanding on 
my part about how to use custom managers.

In our model, we have a Story class, with a boolean 'deleted' 
attribute.  We defined a custom default manager to return all of the 
stories that are not deleted, so that most of the code on the site will 
not have to deal with filtering out the deleted stories (there's another 
custom manager called with_deleted for those code paths that need to see 
the deleted stories as well).

The problem comes up when deleting a user.  The deletion code iterates 
the user's stories to delete them, but uses the default manager, so 
stories with deleted=1 are not found.  So (ironically) any stories 
marked with the 'deleted' field are not actually deleted when the user 
is removed.  This prevents the user from being deleted, because the 
story table still has a foreign key reference to the user table, and 
MySQL prevents the user record from being deleted.

The docs for custom managers say: "it's generally a good idea for the 
first Manager to be relatively unfiltered".  But it seems that any 
filtering in the default manager will interfere with cascading deletes. 

So is this something to fix in the ORM, or have I lost the scent on 
custom managers?

--Ned.

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Initial data hooks: management.install vs. management.syncdb

2006-09-04 Thread Ned Batchelder


I've been experimenting with the new test frameworks, and am very 
excited about their potential.

I've hit a snag, and am wondering if it reveals a flaw in the current 
management.py, or if there is something I don't understand yet (most 
likely the latter).  Russ's test runner uses management.syncdb() to 
create the test database.  I've hooked the post_syncdb signal to create 
my initial data.  It works great.

I've also tried Jason's nose-django plugin, and it uses 
management.install(app) to create the test database.  This doesn't fire 
a signal I can hook, so I wasn't able to create my initial data.

But the difference between the two made me wonder if management.py could 
use some clarification.  Which is the proper method to create the 
database?  Do we need both?  If so, should both fire the same event so 
that initial data can be created in a single way? Sorry to have so many 
questions and so few answers. Perhaps there's a simple path here that I 
have not found?

--Ned.

-- 
Ned Batchelder, http://nedbatchelder.com

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---

Re: name of test database

2006-09-04 Thread Ned Batchelder





Matthew Flanagan wrote:

  On 04/09/06, Russell Keith-Magee <[EMAIL PROTECTED]> wrote:
  
  

On 9/4/06, Ned Batchelder <[EMAIL PROTECTED]> wrote:


  
Partly on this topic:  I would very much like to use sqlite in-memory
  

databases for testing, even though I use MySQL for deployment.  The speed
difference is 10x.  One way to do this is to have a TEST_DATABASE_ENGINE
setting, and add logic to create_test_db().


  Thoughts?

  

If there were a TEST_DATABASE_ENGINE setting, it would imply that there were
analogs of all the other DATABASE_ settings, too. I can't say I'm crazy
about this idea. 'Deploy under MySQL, but test under PostgreSQL' isn't a
particularly compelling use case for me. IMHO, testing for one database but
deploying onto another sort of defeats the purpose of testing.

However, when you are developing unit tests, having a fast turnaround on
test execution is certainly desirable; using SQLite in-memory is certainly
one way to get this fast turnaround.

Rather than having TEST_DATABASE_ENGINE (which implies that ANY database
engine could be used), I would be inclined to have a TEST_IN_MEMORY setting,
which forces all tests to be run in SQLite, using an in memory database.

Furthermore, I would be inclined to make this a command line option, rather
than a permanent setting. IMHO, tests should always be executed against the
_actual_ database that will be used during deployment. Although there
_shouldn't_ be any differences between databases, there occasionally will be
- either due to kinks in Django, or because you are deploying custom SQL.
Providing a command line option would provide a workaround to get fast
development turnarounds without compromising the default (read - preferable)
testing conditions.

Opinions?

Yours,
Russ Magee %-)



  
  
I don't see the need for Django specific TEST_* settings when it is
simple enough to do this in myapp/testsettings.py:

from myapp.settings import *
DATABASE_ENGINE='sqlite3'
DATABASE_NAME=':memory:'

and use ./manage.py --settings=myapp.testsettings.


matthew
  

An excellent suggestion!
-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: name of test database

2006-09-03 Thread Ned Batchelder





Russell Keith-Magee wrote:
On 9/4/06, Ned Batchelder
<[EMAIL PROTECTED]>
wrote:
  
  

Partly on this topic:  I
would very much like to use sqlite in-memory
databases for testing, even though I use MySQL for deployment.  The
speed difference is 10x.  One way to do this is to have a
TEST_DATABASE_ENGINE setting, and add logic to create_test_db().

Thoughts?


  
  
  
If there were a TEST_DATABASE_ENGINE setting, it would imply that there
were analogs of all the other DATABASE_ settings, too. I can't say I'm
crazy about this idea. 'Deploy under MySQL, but test under PostgreSQL'
isn't a particularly compelling use case for me. IMHO, testing for one
database but deploying onto another sort of defeats the purpose of
testing. 
  
However, when you are developing unit tests, having a fast turnaround
on test execution is certainly desirable; using SQLite in-memory is
certainly one way to get this fast turnaround.
  
Rather than having TEST_DATABASE_ENGINE (which implies that ANY
database engine could be used), I would be inclined to have a
TEST_IN_MEMORY setting, which forces all tests to be run in SQLite,
using an in memory database. 
  
Furthermore, I would be inclined to make this a command line option,
rather than a permanent setting. IMHO, tests should always be executed
against the _actual_ database that will be used during deployment.
Although there _shouldn't_ be any differences between databases, there
occasionally will be - either due to kinks in Django, or because you
are deploying custom SQL. Providing a command line option would provide
a workaround to get fast development turnarounds without compromising
the default (read - preferable) testing conditions.
  
  
Opinions?

I'm not sure there's much value in Django having a "preferred" way for
developers to run their tests.  The most valuable tests are the ones
that are run day-to-day, and the ORM does a good job isolating the
Python code from the details of the database, so allowing fast tests
with excellent verisimilitude to the real environment sounds like a
good thing to me.  That said, I understand the desire to cut down on
too many infrequently-used settings.  

A TEST_IN_MEMORY setting is a good compromise, though I would prefer it
to be settable in the settings file, to reduce the need for extra
scripts to get all members of the team to perform tests the best way.

--Ned.
-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

Re: name of test database

2006-09-03 Thread Ned Batchelder





Partly on this topic:  I would very much like to use sqlite in-memory
databases for testing, even though I use MySQL for deployment.  The
speed difference is 10x.  One way to do this is to have a
TEST_DATABASE_ENGINE setting, and add logic to create_test_db().

Thoughts?

--Ned.

Adrian Holovaty wrote:

  On 9/1/06, Russell Keith-Magee <[EMAIL PROTECTED]> wrote:
  
  
In Michael's case, it sounds like test_myapplication would never be a viable
database name (due to some name clash). Given that he knows this, and it
will be a persistent condition, asking him to type './manage.py
--test_db_name="no_clash_db_name" test' every time he runs
his test suite seems a little onerous. He isn't expected to do the same for
DATABASE_NAME - a setting lets him configure his preferred name permanently
on a per-app basis.

I suspect most users will never need the setting - the default naming policy
should be enough. However, the same could be said for other settings, too
(e.g., EMAIL_SUBJECT_PREFIX)

As a bonus, the TEST_DATABASE_NAME setting provides a clean mechanism to
avoid running the Django system tests in an application's test database.

Does that sound reasonable?

  
  
OK, cool -- sounds reasonable. I just wanted to make sure it was
essential / thought-through.

Adrian

  


-- 
Ned Batchelder, http://nedbatchelder.com


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups "Django developers" group.  To post to this group, send email to django-developers@googlegroups.com  To unsubscribe from this group, send email to [EMAIL PROTECTED]  For more options, visit this group at http://groups.google.com/group/django-developers  -~--~~~~--~~--~--~---

83 matches

Mail list logo