Re: TEMPLATE_CHARSET

2007-04-10 Thread Gabor Farkas

Malcolm Tredinnick wrote:
> On Mon, 2007-04-09 at 18:56 +0400, Ivan Sagalaev wrote:
>> As implemented now in unicode branch templates are loaded from files 
>> stored only in utf-8 (as far as I can read the code). However there's a 
>> problem with legacy template files that are stored in one-byte 
>> encodings. This is unfortunately not a rare thing and happen to raise 
>> offenses from developers who we are forcing effectively recode all their 
>> template files (and may be change text editors etc.)
>>
>> May be a TEMPLATE_CHARSET setting will be useful?
> 
> Yeah, I wondered about this. I was kind of hoping it wouldn't be an
> issue, but people will insist on using non-portable encodings in their
> files even in the 21st century. :-(
> 
> I'm not so much worried about the one-off conversion (after all, it's
> for those peoples' benefit that we're doing this) as much as filesystems
> that store in a particular encoding by default. There's no reliable,
> non-expensive way to automatically detect the file encoding, so it
> probably has to be done with a setting.
> 
> TEMPLATE_CHARSET is not the right name, though. Templates aren't the
> problem: the filesystem encoding is. So maybe FILESYSTEM_ENCODING or
> something explicit like that. We'll need to graft it into each
> filesystem-based template loader and it defaults to utf-8.
> 

python uses sys.getfilesystemencoding to detect the filesystem's 
encoding, but it's not perfect, because for example on linux, there 
isn't any guarantee that this encoding (as far as i know python is 
guessing it based on your locale) is used in your template file.


but what i don't understand, is: why is this an issue? can't we assume 
that the developer knows the encoding of his own files? and if he knows 
them, he should be able to easily recode them to utf-8, for example.

i'm personally for the one-encoding-to-rule-them-all solution btw :)
the only issue i have with it is that if we make utf-8 mandatory,
will it be still easy/possible to generate html files in non-utf-8?

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: newforms and unicode/bytestrings

2007-04-06 Thread Gabor Farkas

Ivan Sagalaev wrote:
> Malcolm Tredinnick wrote:
>> I applaud Adrian's attempts to be unicode-aware from the get go, but
>> it's also cost a bit of sleep lately trying to debug things. Having
>> spent most of my free time over the last three days looking at this
>> exact set of problems, I'm beginning to think we should just bite the
>> bullet and move to unicode everywhere (and assuming UTF-8 encoded
>> bytestrings for the rest) right now. I've been making the changes in a
>> sandbox this afternoon and it will take most of tomorrow to get a sense
>> of how well it's working, I think.
> 
> +∞  !!!
> 
> Actually I'm working with Django now full-time and want to help in 
> making unicode in Django just happen. On my first day at the new job 
> I've immediately encountered two people complaining about these problems 
> on the verge of MySQL DB driver and serialization framework.
> 
> I remember Gábor Farkas had a plan for unicodification. Gábor is it 
> still valid? If not I could elaborate something myself.
> 

hi,

depends on how you define "valid" :)

i did some "tests", converted the request-handling and most of the 
template-rendering, but i don't think it makes sense to use the patches 
i did at that time, because:

- i probably lost them already :)
- it was very easy to do
- they wouldn't apply to trunk cleanly

but i am also +1000 for this change. switching to full unicode is a lot 
less risky than trying to make the unicode-newforms-with-bytecode-django 
  chimera work correctly.

so, if the unification process starts, i'm also eager to help with it.

let's hope Malcolm "branch" will work until the deadline :)

but, even if it not, i think full django-unification is worth ANY cost.


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Status update on newforms-admin branch

2007-02-02 Thread gabor

Adrian Holovaty wrote:
> 
> I've
> also taken the opportunity to refactor some admin-specific options --
> raw_id_admin and prepopulate_from -- so that they're defined in the
> "class Admin" rather than in the model fields themselves.

with this change to the raw_id_admin, what is the recommended approach,
when i want to use a ChangeManipulator (the automatic one generated from 
the model), but i want to do it the raw_id_admin way, otherwise the 
Manipulator loads in every possibly-related object, when doing the 
validation.

up to now i simply added raw_id_admin = True to the model's field, but 
now i do not know how it should be handled...i realize that the 
new-admin is probably using the newforms... and i do not know how 
raw_id_admin is handled there

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Moving towards Django 1.0

2007-01-13 Thread gabor

Jacob Kaplan-Moss wrote:
> * Forms: the newforms library is coming along nicely.  There's some work that 
> remains, the bulk of which lies in converting the admin to use newforms 
> instead of manipulators. At that point, the transition can really be called 
> complete.


it seems that noone mentioned unicode in this discussion yet...

does this mean, that the plan is to have django-1.0 still use bytestrings?

because if yes, then i would recommend to "downgrade" newforms to 
bytestrings too.

the reason is consistency...

if i'm not mistaken, newforms works internally in unicode, and also 
outputs unicode... which is quite nice, but the problem is, the rest of 
django is in bytestrings...

so, at the end, in a django-view:

- GET/POST is in bytestrings
- newforms outputs unicode
- the templates output bytestrings
- the db-api outputs bytestrings

it is true, that with various hacks the newforms can be used from the 
bytestring-based templates, and also they can save models to the 
bytestring-based db-api. but it is not nice.


the other way is to switch django to unicode completely.

from the past discussion it seems, that many people wanted to work on it 
  , patches were created, so i think the unicode-conversion could happen 
in a reasonable time.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: DATABASE_CHARSET setting, once and for all

2006-12-15 Thread gabor

Adrian Holovaty wrote:
> I recently implemented the form_for_model() helper function in
> django.newforms, which returns a Form class for a given Model class.
> It has a create() method, which actually creates the object from form
> data. This is the replacement for automatic AddManipulator classes.
> 
> But there's a problem -- the newforms library returns all data in
> Unicode, whereas the PostgreSQL psycopg1 database backend only accepts
> bytestrings and does not automatically convert Unicode to bytestrings
> (see http://code.djangoproject.com/ticket/3115 ). As a somewhat
> related issue, the ticket http://code.djangoproject.com/ticket/952
> proposes a DATABASE_CLIENT_CHARSET setting. This would specify the
> charset used by your database. Currently Django assumes MySQL uses
> utf-8 in the database, and (if I remember correctly) assumes other
> databases are using the encoding specified by DEFAULT_CHARSET.
> 
> Does anybody have comments on that patch before I commit it?
> 

+1

gabor

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Django Templates and BOM (byte order marks)

2006-12-08 Thread gabor

Sergey Kirillov wrote:
> Greetings,
> 
> While researching strange IE behaviour at some pages of my
> Django-powered site I've found that Django does not handle BOM marks
> correctly.
> 
> For instance I have following templates (!UTF_8_BOM! = EF BB BF):
> 
> --- base.html
> !UTF_8_BOM! Transitional//EN"
>"http://www.w3.org/TR/html4/loose.dtd;>
> blah-blah-blah
> ---
> 
> and
> 
> --- page.html
> !UTF_8_BOM!
> {% extends "base.html" %}
> blah-blah-blah
> ---
> 
> If I will render page.html output will contain TWO byte order marks:
> !UTF_8_BOM!!UTF8_BOM! Transitional//EN" ...
> 
> 
> 
> Of course I can fix this by introducing custom middleware, but I think
> problem should be handled at template loader level.
> 
> 
> Any comments?


i experimented a little, and here's what i found:

the basic "problem" is that django simply treats the templates as 
byte-strings. for him even the BOM is simply a 3bytes-long text.

so, for example, in the following case:

==base.txt==
bbb
bbb
bbb



==inherit.txt===
xxx{% extends "base.txt" %}


if you render inherit.txt, you get


xxxbbb
bbb
bbb


as you see, the text before the extends-tag is kept.

and, in your case that text is the BOM.

so technically django behaves correctly (well, let's say consistently :)

even if we could change it, what would you propose?

i think the best solution would be to:
- strip the BOM, and remember that there was a BOM
- at the end, when the final rendered template is emitted, add the BOM 
if it was used in the input templates

but this would mean that we have to "unicode-interpret" the template, 
and i don't think that will happen before django goes completely unicode.

so, imho your best way of action is to strip the BOM from your templates

gabor

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Have a look at django.newforms

2006-12-05 Thread gabor

Adrian Holovaty wrote:
> On 12/1/06, Gábor Farkas <[EMAIL PROTECTED]> wrote:
>> what about the following change?:
>>>  if not isinstance(output, basestring):
>>> -return str(output)
>>> +try:
>>> +return str(output)
>>> +except UnicodeEncodeError:
>>> +return unicode(output).encode(settings.DEFAULT_CHARSET)
>>>  elif isinstance(output, unicode):
>>>  return output.encode(settings.DEFAULT_CHARSET)
>> from the python docs (http://docs.python.org/lib/built-in-funcs.html)
>> unicode() technically should not work in this case, but it somehow works :)
>>
>> this change would not introduce any backward incompatibility,
>> because this new code is only triggered when the old one failed.
>>
>> another (independent from this change) enhancement would be to add an
>> __unicode__ method to the Form object.
> 
> This is an interesting problem. That template fix would be OK by me,
> but it's sort of a hack. I think we're going to run into similar
> issues with Form.__str__() returning a Unicode object. 

 > Maybe, as you
> suggest, Form.__str__() should return a bytestring according to
> DEFAULT_CHARSET, and we could add a Form.__unicode__() that would
> return it as Unicode.

hmmm.. i agree :)

> 
> Thoughts? And would that approach mix well with the upcoming Python
> 3000 changes? I thought I read something about __unicode__ and __str__
> merging...

strings in py3000 will be unicode strings (there will be byte-arrays to 
handle byte-related things),
so if __str__ keeps his "return a string-based representation of the 
object" function, he will probably return unicode strings (because there 
won't be any other strings)

gaobr

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Have a look at django.newforms

2006-11-30 Thread gabor

Adrian Holovaty wrote:
> Hi all,
> 
> The django.newforms library is getting more solid by the day, and I'm
> using it for some personal and work projects. Real-world use has
> helped me find and fix problems with it, and I'm pretty happy with how
> things have turned out.
> 
> So, then, I'd encourage everybody to play around with django.newforms
> and post a message if you have any issues or thoughts.

i tried it out yesterday, and it's very clean and intuitive.

my only worry is regarding the unicode-behavior

i created a simple form, like:

> class UniInput(Form):
> text = CharField(max_length = 200)

but if i want to render it in a template and it contains non-ascii text, 
it fails in /home/gabor/src/django/django/template/__init__.py,
line 745, UnicodeEncodeError.

the code there calls str() on the form, and it's a problem because it 
calls Form.__str__, which in turn calls as_table, which returns unicode 
data.
which is then converted to bytestring, by the default-charset (ascii),
and that fails.

currently i solved it by sending the form as 
"form.__str__().encode('utf-8')" into the context, but that's not that nice.

one solution would be to special-case Form-object-handling in the 
template-code. currently it handled unicode and bytestrings, maybe it 
could also handle Form instances.

but the real problem imho is that Form.__str__ returns an unicodestring.
the python docs say that "The return value must be a string object." 
which i'm not sure how should be interpreted :)

maybe Form.__str__ could encode the unicode data into a bytestring using 
DEFAULT_CHARSET?

generally, before django is completely converted to unicode, how do you 
plan to integrate the forms (their input and their output) into django?

gabor

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Model quoting error : Postgresql 8.1

2006-11-19 Thread gabor

Rory Campbell-Lange wrote:
> Dear Developers
> 
> The error:
> 
> INSERT INTO "data_sector" 
> ("name","localname","created","modified","country_id") 
>VALUES 
> (Administrative Offices,'','2006-11-18 
> 21:36:31.145400','2006-11-18 21:36:31.145474',1)
> 
> 'Administrative Offices' isn't quoted. I believe this is because the
> 'Administrative Offices' string is a unicode string (u'Ad...') and that
> the psycopg module can't handle unicode strings.

yes, it's a bug in psycopg1. it does not escape unicode strings.
it should probable be solved by the django-layer, but it's planned to 
switch django completely to unicode, so this issue will probably be 
solved that time.
> 
> This is a problem reported in bug #305. I am using the work-around
> provided in that report to encode unicode strings to utf-8, but it seems
> inelegant.
> 
> I've installed psycopg2, as I believe this does not suffer from this
> issue. However I cannot find a straightforward way of setting django to
> use psycopg2.


DATABASE_ENGINE = 'postgresql_psycopg2'

gabor

> 
> Thanks for any help;
> Rory
> 
> 
> Background:
> 
> data.models.Sector definitions:
> 
> class Sector (models.Model, General):
> name  = models.CharField('Project Sector Name', blank = False, 
> maxlength = 200)
> localname = models.CharField('Local Sector Name'  , blank = False, 
> maxlength = 200)
> created   = models.DateTimeField(auto_now=True)
> modified  = models.DateTimeField(auto_now_add=True)
> country   = models.ForeignKey(Country)
> 
> Source of information I am trying to load:
> 
> secs = [
> [1, None, u'Administrative Offices'], 
>   [2, None, u'Higher Education'], 
>   [3, None, u'Hospitals'], 
>   [4, None, u'Hotels'], 
>   [5, None, u'Schools'], 
>   [6, None, u'Sports']
>  ]
> 
> This is a code snippet from the shell:
> 
>>>> c = Country.objects.get(id=1)
>>>>
>>>> for s in secs:
> ...newsector = Sector(name=s[2], localname='', country = c)
> ...newsector.save()
> ... 
> Traceback (most recent call last):
>   File "", line 3, in ?
>   File 
> "/usr/lib/python2.4/site-packages/Django-0.95-py2.4.egg/django/db/models/base.py",
>  line 203, in save
> ','.join(placeholders)), db_values)
>   File 
> "/usr/lib/python2.4/site-packages/Django-0.95-py2.4.egg/django/db/backends/util.py",
>  line 12, in execute
> return self.cursor.execute(sql, params)
> psycopg.ProgrammingError: ERROR:  syntax error at or near "Offices" at 
> character 105
> 
> INSERT INTO "data_sector" 
> ("name","localname","created","modified","country_id") VALUES (Administrative 
> Offices,'','2006-11-18 21:36:31.145400','2006-11-18 21:36:31.145474',1)
> 
> 
> Psycopg version : python-psycopg 1.1.21-10
> Django version  : 0.95
> OS  : Debian Testing
> 
> 


--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: newforms, unicode

2006-11-17 Thread gabor

Adrian Holovaty wrote:
> On 11/16/06, gabor <[EMAIL PROTECTED]> wrote:
>> - what is the plan, how should incoming data (GET or POST) handled?
>> some possibilities:
>> [...]
>> - we assume it's DEFAULT_CHARSET. we do not failover (means that if that
>> data is not in DEFAULT_CHARSET, the newforms code will (because of
>> "unicode(data,DEFAULT_CHARSET) raise the usual UnicodeDecodeError)
>> [...]
>> - we will make sure that we never raise such an error. that would mean,
>> that, for example:
>> - we try DEFAULT_CHARSET
>> - if we fail, we try  (probably utf-8)
>> - if we fail, we try 
>> - if we fail, we use ISO-8859-1 (which cannot fail)
> 
> I'd been assuming the first strategy -- no failover at all for data
> not in DEFAULT_CHARSET -- but I'm glad you brought this up, because
> that strategy is quite rigid and could cause hard-to-debug problems.
> Then again, the alternative (trying multiple charsets, one at a time)
> seems like overkill. What does everyone think?

btw. i'd prefer the first strategy.

i agree that it's rigid, but how would it cause "hard-to-debug" problems?

maybe i'm looking at the problem from a bit different point of view,

because i had this problem many times when trying to come up with a 
solution for unicode-django.

generally the question is this:

- the GET/POST-data comes in as byte-strings. how should we convert them 
to unicode strings?

imho the developers will simply have to learn that there is no easy 
solution when dealing with unicode, and if they send into the view data 
in the wrong encoding (read: not DEFAULT_CHARSET), then it will fail.

also, if we raise an exception, the developer will at least be informed, 
that the data he is sending in is wrong. if we will failover to 
ISO-8859-1, he might not find out about this problem for a long time.

btw. i am not sure when the unicode-conversion (and the raising of the 
exception) should happen...

at the beginning:

>form = ContactForm(request.POST)

or only when calling is_valid

>   if form.is_valid(): 


gabor

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



newforms, unicode

2006-11-16 Thread gabor

hi,

i think it would be a good idea, to define what the "unicode policy" of 
newforms should be.

i got the idea after seeing changeset 4076

i understand that this is still work-in-progress, but we still should 
clearly define how we handle incoming data.

so, some question:

- what is the plan, how should incoming data (GET or POST) handled?
some possibilities:

- we assume it's DEFAULT_CHARSET. we do not failover (means that if that 
data is not in DEFAULT_CHARSET, the newforms code will (because of 
"unicode(data,DEFAULT_CHARSET) raise the usual UnicodeDecodeError)

or

- we will make sure that we never raise such an error. that would mean, 
that, for example:
- we try DEFAULT_CHARSET
- if we fail, we try  (probably utf-8)
- if we fail, we try 
.
.
.
- if we fail, we use ISO-8859-1 (which cannot fail)

so, which direction should be taken?

gabor

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



postgresql, duplicate indexes?

2006-11-12 Thread gabor

hi,

in certain cases, django-postgresql creates duplicate indexes...


imagine the following model:

 > class Item(Model):
 >text = CharField(maxlength=200,unique=True, db_index = True)

when installing it, it generates the following sql: (manage.py sqlall)


 > BEGIN;
 > CREATE TABLE "y_item" (
 > "id" serial NOT NULL PRIMARY KEY,
 > "text" varchar(200) NOT NULL UNIQUE
 > );
 > CREATE UNIQUE INDEX y_item_text ON "y_item" ("text");
 > COMMIT;


and the result is the following db schema:

> boo=# \d y_item
>  Table "public.y_item"
>  Column |  Type  |  Modifiers 
>  
> ++-
>  id | integer| not null default 
> nextval('y_item_id_seq'::regclass)
>  text   | character varying(200) | not null
> Indexes:
> "y_item_pkey" PRIMARY KEY, btree (id)
> "y_item_text" UNIQUE, btree (text)
> "y_item_text_key" UNIQUE, btree (text)

now, if you check the indexes, ithe "text" column is indexed twice...

the reason is that because it is UNIQUE, postgres automatically creates 
an index for it.

and then, because it has db_index = True, it gets a second index from 
django.

i'm no postgresql expert, but it seems to me that having 2 indexes on 
the same column is a bad thing (because when writing into this table, 
the index will have to be calculated twice).

(or are they "different" indexes?)

so, should i open a bugreport (ticket) about this?

or is this intentional?

also, in this case, the "workaround" is simple. simply remove the 
db_index=True.
but it's not that simple always.

for example imagine an unique SlugField. in that case the only possible 
fix is to manually delete the redundant index from the db.

gabor

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Branch Merges?

2006-11-07 Thread gabor

Adrian Holovaty wrote:
> On 11/7/06, Rob Hudson <[EMAIL PROTECTED]> wrote:
>> Adrian Holovaty wrote:
>>> That's exactly what we're using the branches for: Things get broken on
>>> the branches as branch owner merge in new features. Eventually, bugs
>>> get worked out, and they do a code freeze prior to the merge to make
>>> the branch as stable as possible.
>> I suppose the disadvantage to this is getting users to test branches.
> 
> That's a good point. Maybe we could do a better job of this by
> highlighting the currently developed branches on the main Django
> download page, rather than hiding them on the wiki, which I'm still
> convinced many people don't know about. Thoughts?

(+1000 on the keep-the-trunk-stable approach)

i think it's a good idea, but then we also have to make sure that the 
documentation of the branch is up-to-date, so that it's clear if it's is 
testing-ready or not, and what it does offer.

gabor

--~--~-~--~~~---~--~~
 You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---



Re: Database race conditions when using multiple processes

2006-10-16 Thread gabor

James Bennett wrote:
> On 10/16/06, gabor <[EMAIL PROTECTED]> wrote:
>> would you use something in the db? (from what i know about transactions
>> (very little :), they "solve" the potential conflicts by simply
>> reporting an error-condition to one of the "writers", so then he has to
>> retry. but is there some simply (multiprocess-usable) way to lock (as in
>> "wait until this guy is finished")?
> 
> http://www.postgresql.org/docs/8.1/static/sql-lock.html
> 

thanks a lot


(and again something that would be relatively easy to find with a 
google("postgresql lock"). next time i will do the googling before 
asking, i promise (/me makes a mental note of it AGAINNN) :)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Database race conditions when using multiple processes

2006-10-16 Thread gabor

James Bennett wrote:
>  For truly critical
> operations, I'd probably just go ahead and grab an exclusive lock on
> whatever I needed, and force other transactions to wait.

how would you that?

would you use something in the db? (from what i know about transactions 
(very little :), they "solve" the potential conflicts by simply 
reporting an error-condition to one of the "writers", so then he has to 
retry. but is there some simply (multiprocess-usable) way to lock (as in 
"wait until this guy is finished")?

or would you rather do it with something completely different? a 
temporary lock-file in a temp-directory perhaps?

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Database race conditions when using multiple processes

2006-10-16 Thread gabor

Thomas Steinacher wrote:
> gabor wrote:
>> Thomas Steinacher wrote:
>>> I already included
>>> "django.middleware.transaction.TransactionMiddleware" in my
>>> MIDDLEWARE_CLASSES (as described on
>>> http://www.djangoproject.com/documentation/transactions/#tying-transactions-to-http-requests)
>>> and it didn't help.
>>>
>> you mean it did not help with the money-transfer code?
>>
>> gabor
> 
> It did not help with the code I posted that was using get_or_create(),
> because I didn't use unique_together... I don't think it would help
> with the money-transfer code.
> 
> The documentation says:
> 
> "It works like this: When a request starts, Django starts a
> transaction. If the response is produced without problems, Django
> commits any pending transactions. If the view function produces an
> exception, Django rolls back any pending transactions."
> 
> The problem is that the code doesn't produce an exception, so Django
> will not roll back anything...
> 

ok, then maybe this is a misunderstanding, so let's restart the discussion:

what is your question? :)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Database race conditions when using multiple processes

2006-10-16 Thread gabor

Thomas Steinacher wrote:
> On Oct 16, 2006, at 8:46 PM, gabor wrote:
>>> What, if two Django processes transfer money from account A to account
>>> B simultaneously?
>> these situations are (afaik) handled using transactionssimply start
>> a transaction at the beginning, and commit/rollback at the end...
> 
> I already included
> "django.middleware.transaction.TransactionMiddleware" in my
> MIDDLEWARE_CLASSES (as described on
> http://www.djangoproject.com/documentation/transactions/#tying-transactions-to-http-requests)
> and it didn't help.
> 

you mean it did not help with the money-transfer code?

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Database race conditions when using multiple processes

2006-10-16 Thread gabor

Thomas Steinacher wrote:

> I know that in this specific case it can be enforced at database level
> using the unique and unique_together properties, but how would you
> handle e.g. this scenario?
> 
> 1  def transfer_money(from_id, to_id, amount)
> 2from = models.BankAccount.objects.get(id=from_id)
> 3to = models.BankAccount.objects.get(id=to_id)
> 4from.balance -= amount
> 5to.balance += amount
> 6from.save()
> 7to.save()
> 
> What, if two Django processes transfer money from account A to account
> B simultaneously?
> 

these situations are (afaik) handled using transactionssimply start 
a transaction at the beginning, and commit/rollback at the end...

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: bump: give Gabor (or somebody else) a unicode branch in svn

2006-10-13 Thread gabor

Adrian Holovaty wrote:
> On 10/13/06, Ivan Sagalaev <[EMAIL PROTECTED]> wrote:
>> Max Derkachev wrote:
>>> There were a lot of words about unicodification of Django, but things
>>> has not been moved a bit further.
>> Or at least it would be nice to hear from someone of the core team why
>> it can't be done (either right now or at all). I remember Jacob (I
>> think) has said once along the lines that he's not very excited about
>> unicodification since because of too little gain from too much effort.
>> But the branch will be maintained by other people so why not try?
> 
> Hey there,
> 
> I'm very interested in getting this done, but we've got quite a few
> branches open at the moment. I think we should focus on merging at
> least one of these branches before opening another one, for the sake
> of everybody's sanity.

hi,

+1

btw. i'm slowly (VERY slowly :) working on the unicodification 
(currently on the template system), so it is not halted (just very slow :)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: CharFields and children defaulting to emtpy string

2006-09-16 Thread gabor

DavidA wrote:
> 
> Malcolm Tredinnick wrote:
>> When I first started using Django, this drove me nuts.
> 
> I had the same reaction, but also understand there isn't a great way to
> handle it. In Microsoft's SQL  Enterprise Manager, you can enter Ctrl-0
> in a field and it will set that field to NULL. I'm thinking about doing
> something like this in one of my Django forms since I need to support
> both an empty string _and_ a NULL input so I want an explicit way to
> differentiate between the two. A control character doesn't work well
> for browser input, but I may end up treating an empty string as NULL
> and force the user to put "" if they really want a blank string (which
> in this case is the exception, not the norm).

assuming that you want to differentiate between:

- the user did not fill in the data
- the user's input was ""

wouldn't it be better to represent this in html as a checkbox+an input 
field?

and by default have the checkbox unselected, and the input-field disabled.
and then if the user checks the checkbox, enable the field.

and then in the view code, simply check the checkbox's value and set the 
field to null or to what-was-in-the-input-field.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Local unicode-related fixes

2006-09-16 Thread gabor

Malcolm Tredinnick wrote:

> 
> [I'm a leading advocate for the "give Gabor a unicode branch in svn"
> school, btw, so that you and he and others can go nuts and just do it.]
> 

+1 :-)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: CharFields and children defaulting to emtpy string

2006-09-11 Thread gabor

Gary Wilson wrote:
> using the example model:
> 
> class Name(models.Model):
> name = models.CharField(maxlength=100)
> 
> when making a new Name object...
>>>> a=Name()
>>>> a.save()
>>>> a
> 
>>>> a.name
> ''
> 
> The name field does not have blank=True or null=True set, but yet I am
> able to create a new Name object using no parameters (because
> CharFields and children are defaluting to empty string).  Is this the
> intended behavior?
> 

hi,

yes, (unfortunately) it is.

the models do not validate their input.

you will have to use a manipulator (Name.AddManipulator) to validate 
your input (and then save the object).

making the models validation-aware is planned, but not done yet.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Integrating Django and SQLAlchemy

2006-08-29 Thread gabor

Robin Munn wrote:
> 
> The notes on implementation that Adrian posted pretty much match what
> I'm thinking at this point. The plan is to make this 100% API
> compatible (if possible -- you never know what will turn up once you
> start implementing some idea), so that existing code doesn't need to
> change. But if you want access to the underlying SQLAlchemy objects to
> do something like a nine-table query involving five LEFT OUTER JOINs
> and two subselects, well, you should be able to do that as well.

hi,

this is great news...

could you show some example code how would it work?
like
without sqlalchemy you would have to do this in raw sql, but with 
django+sqlalchemy, you could do that
things like that?

because i checked the sqlalchemy tutorial, and i see that it is a 
powerful framework... i just don't currently see how it would interact 
with django


please don't take this wrong, i very much welcome this django/sqlalchemy 
integration... i'm just curious how exactly would it work... how will i 
be able to go "deeper" (from django-level into sqlalchemy level) when 
needed...

or ...well..maybe it's too early to ask questions like this :)

thanks,
gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: [Fw]The Python Web Framework

2006-08-22 Thread gabor

JP wrote:
> James Bennett wrote:
>> On 8/22/06, Karl Guertin <[EMAIL PROTECTED]> wrote:
>>> SA provides:
>>> * connection pooling - since you asked about it, I'll quote from the SA 
>>> docs:
>> This is why I said "what comes to mind when I think 'database
>> connection pooling' isn't something I think belongs in Django."
>>
>> Maintaining in-process connection pools in the framework doesn't, in
>> my experience, provide enough benefit to justify the hassle. I'd
>> rather talk to something like pg_pool which maintains an external
>> connection pool.
> 
> I think the problem here is competing definitions of the term
> 'connection pool'. You're using 'connection pool' to mean a connection
> that may be transparently directed to one of N databases (say where
> records 1-100 are on database machine A and 101-20 are on
> database machine B, or you're writing to A and reading from A and B), I
> think. And I think that Karl and Christopher are using 'connection
> pool' to mean a collection of N persistent connections to the SAME
> database, where threads in a web app can check out and use connections
> from the pool, so that each thread need not make and hold its own
> connection.
> 
> So I agree, django's ORM doesn't need pools (definition 1), but it does
> need pools (definition 2) to help it scale better in some environments
> and to reduce request startup time.
> 

hmmm..so am i correct when i say, that in a non-multithreaded web-app 
definition-2-pools are not needed/ do not help?

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: django unicode-conversion, beginning

2006-08-21 Thread gabor

Adrian Holovaty wrote:
> On 8/8/06, gabor <[EMAIL PROTECTED]> wrote:
>> i think unicodizing django can be done in 4 easily separated steps/parts:
>>
>> 1. request/response
>> 2. templating-system
>> 3. database-system
>> 4. "overall unicode-conversion". this is mostly about replacing
>> bytestrings with u"bla" in the code, and switching __str__ to __unicode__
>>
>> my biggest problem currently is, that i do not know how to continue...
>> should i just write more and more patches to increase the
>> "unicode-coverage" to more parts of django? or maybe a more coordinated
>> approach would be better?
> 
> Hey gabor,
> 
> Sorry for the slow response on this -- I'm just now wading through a
> couple of weeks' worth of django-users and django-developers messages.
> This patch is a great step forward!
> 
> Are you interested in a Subversion branch devoted to Unicoding Django?
> Let me know...
> 

(to make sure my original response is not caught up in a spam-filter or 
such, sending this to the list too)


hi,


yes, i'm interested :)

cannot really promise how long it will take to convert the whole django 
to unicode, but will try. it's not hard. as i wrote, the changes are 
simple, it's just that many changes have to be done.


thanks,
gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: django unicode-conversion, beginning

2006-08-16 Thread gabor

Jeremy Dunck wrote:
> On 8/16/06, Bill de hÓra <[EMAIL PROTECTED]> wrote:
>> Now. Most (all?) browser UAs sniff the content to second guess the media
>> type. They don't much pay attention to Content-Type (I think maybe IE
>> ignores it altogether). The problem for this example is they might be
>> doing something similar for character encodings declared on the form
>> page's GET request. Browsers do this because so much served content is
>> mislabelled (eg feeds served as text/html and video as text/plain).
> 
> IE doesn't totally ignore it.  I just does some horrible, wrong things
> while considering it.
> http://blogs.msdn.com/ie/archive/2005/02/01/364581.aspx
> http://msdn.microsoft.com/workshop/networking/moniker/overview/appendix_a.asp
> 
> Ian Hickson says contenttype is dead:
> http://ln.hixie.ch/?start=1144794177=1
> http://ln.hixie.ch/?start=1154950069=1
> 

hmmm.. sad to hear that.. but it hopefully does not affect the 
django-unicode issue too much...

currently my plan is to have the following behaviour:

1. i assume that every GET/POST param comes in encoded as 
settings.DEFAULT_CHARSET, and will decode it accordingly. if it fails, 
then it fails.
- might make an exception and in case of post-data check the 
content-type header of the request, whether it contains any charset stuff
-if you really-really-really need to do some crazy 
is-sent-as-foo-but-has-to-be-treated-as-bar, you can always use the 
raw-postdata and raw-getdata.

2. will render the template in DEFAULT_CHARSET

3. will assume the database is in DEFAULT_CHARSET
- maybe can we somehow ask the db for it's charset?

so, what do you think?
or should we make it possible to have a system with mixed charsets? 
(well, maybe having a different DB_CHARSET and a DEFAULT_CHARSET could 
work. maybe)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: django unicode-conversion, beginning

2006-08-09 Thread gabor

Malcolm Tredinnick wrote:
> A couple of comments on the patch itself. I realise it's only a proof of
> concept at the moment, so take as more things to think about when you
> want to tidy it up:
> 
> (1) A docstring like """needed to workaround the cgi.parse_sql
> unicode-problem""" is not very future-proof. *What* parse_sql unicode
> problem? How will we know if/when it goes away? Either a quick
> description of the problem or a URL if it's tricky and explained
> elsewhere will help people who need to read this code in six months
> time.

ok

> 
> (2) You can't necessarily assume the environment is always in ASCII (or
> maybe you can; see below). For example, my current locale is set to
> en_AU.UTF-8 and I can do
> 
> export foo="€50,00"
> 
> If I'm not careful when parsing os.environ['foo'] this comes out as
> rubbish (I need to do unicode(os.environ['foo'], 'utf-8') or similar).
> 
> Probably some playing around with the locale module to work out the
> right behaviour and getting a few people to test things (e.g. Windows
> vs. Linux vs. Macs, etc) will be necessary. It's also important not to
> go too overboard here, but since arbitrary environment variables can be
> set through Apache, we need to be able to work with that to be
> "correct". Hmm ... what are the restrictions on what webservers can put
> in their config files? Maybe ASCII-only is reasonable. *shrug*
> 

phew... the immortal 
how-tolerant-we-should-be-when-doing-unicode-conversion problems :-)

i generally prefer to do as little guesswork as possible, but in the 
case of the environ-variables it seems we cannot avoid it.. after all, 
it cannot crash when parsing the environ variables, because there's no 
way from the programmer's side to affect them.

so what do you think about the following approach:

try ascii-decoding
if fails, try utf8-decoding
if fails do iso-8859-1-decoding (this cannot fail).

?


but imho this should happen only in "special" cases like 
environ-variables.. for example in get/post params i would prefer to 
raise an exception when the data cannot be en/de-coded using the 
configured charset.

> Maybe more investigation needed here.
> 
> (3) I know there are some software projects apparently using unicodize
> as a word, but ... *shudder*. Using "code" as an analogy, "unicodify"
> would be nicer (nobody uses "codize", I would hope).
> 

ok

> (4) As you go through this process, keep a list somewhere of what people
> need to do to port existing applications across to using this
> functionality. Ideally, the answer would be "not much" and we can cast
> from the default encoding to unicode internally where necessary. But I'm
> sure there will be some changes required, so keeping a list of things to
> watch out for as you go will help people test this for you.
> 

will try.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



django-unicode conversion, first experiments, questions

2006-07-18 Thread gabor

hi,

i looked at the convert-django-to-unicode problem,
did some simple hacks here and there, and got a couple of questions :-)


i did not touch the templating and the orm layer for now, so this is 
about the httpRequest issues...


to quickly go through how the request is processed:
(i'll only talk about wsgi here)

- at the beginning django gets a dictionary (environ), which contains 
all the data about the request
- from this django builds a httpRequest
- finds the suitable view using some regex-matching

that's it for now.

i think we can all agree that the goal is that a developer who uses 
django only meets unicode strings...

we can also agree imho that the request.GET and request.POST 
dictionaries should contain unicode strings (keys and values too).

questions:
1. django publishes the whole environ dictionary as request.META. the 
environ dictionary is a normal byte-string dictionary. so, should we 
convert it to unicode so that the request.META dictionary only contains 
unicode strings?

1.a: if yes, how? some things like QUERY_STRING and REQUEST_METHOD are 
relatively simple how to convert, but what should happen to the other 
server-related stuff like HTTP_X_FORWARDED_HOST? should we convert them? 
if yes, with what encoding? ascii? iso-8859-1?

2. urlconf... should it work on unicode strings? i think yes, because 
later, in the view the parameters should arrive as unicode strings, so 
probably the urlconf-strings should be unicode too. btw. should we 
un-escape the strings that we match in the urlconf (means the strings 
that then are sent to the view as parameters).


i hope with these questions i can start some kind of a discussion to 
better specify what we want from this unicode-conversion.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: ManyToManyField looks broken

2006-06-25 Thread gabor

Malcolm Tredinnick wrote:
> 
> I can no longer cause #1796 or anything like it to fail -- and I had a
> couple of failing examples there. I believe relative imports
> (appname.models, etc) might now work reliably too (experiments suggest
> they do), which means you no longer are reliant on your project name and
> can create portable apps.

could you please try what i described here:
http://tinyurl.com/kkpgw

for me it still fails :-(

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: validate_full never called?

2006-06-17 Thread gabor

Jay Parlar wrote:
> On 6/16/06, gabor <[EMAIL PROTECTED]> wrote:
>> Jay Parlar wrote:
>> currently you have to use manipulators, if you want to validate the data
>> that enters your models, see
>> http://www.djangoproject.com/documentation/forms/
>>
>> later, when validation-aware models will be implemented, this validation
>> step will happen in the model itself.
> 
> I do use manipulators, ie.
> 
> manipulator = xxx.AddManipulator()
> ...
> new_data = request.POST.copy()
> errors = manipulator.get_validation_errors(new_data)
> 
> 
> but it wasn't catching the empty field. I just assumed that the
> 'blank=False' would be used by the manipulator during its validation
> stage.
> 

could you post your model and the relevant part of the view code?

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: validate_full never called?

2006-06-16 Thread gabor

Jay Parlar wrote:
> On 6/16/06, Adrian Holovaty <[EMAIL PROTECTED]> wrote:
>> On 6/16/06, Jay Parlar <[EMAIL PROTECTED]> wrote:
>>> I just noticed today that if I have a CharField primary_key, with
>>> 'blank=False', the system doesn't actually ever validate that the
>>> field isn't empty.
>> The validation stuff isn't finished and isn't intended to be used yet.
>> Move along, nothing to see here. :)
>>
>> Adrian
> 
> Hehe, ok, I can live with that for now. By "validation stuff" though,
> what exactly does that include? Keywords passed to the fields in the
> model? Or anything else I should keep my eyes open for?
> 

currently you have to use manipulators, if you want to validate the data 
that enters your models, see 
http://www.djangoproject.com/documentation/forms/

later, when validation-aware models will be implemented, this validation 
step will happen in the model itself.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Proposal: default escaping

2006-06-14 Thread gabor

Jacob Kaplan-Moss wrote:
> Hi folks --
> 
> So the benefits of automatic escaping are pretty obvious --  
> protection from XSS attacks -- but I'm wary of a few details in the  
> existing proposals.
> 
 > 

i completely agree that before doing such a global change, all 
consequences will have to be examined/specified.
> 
> Given that, I think the best idea is still using a block tag::
> 
>   {% escape %}
> {{ var }}
>   {% endescape %}
> 
> that just seems the most clear to me.

maybe we could try to answer a question:

is it true, that people usually forget to escape dangerous variables?


a) if no (people do not forget):
means people are already using 'escape' when needed. in this case, this 
block-level tag is a welcome addition, because it makes it 
simpler/more-convenient to toggle escaping.


b) if yes (people do forget):
a block level tag will not help. people will forget to use them the same 
way they forget to use the 'escape' filter.

my guess is (b)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



validation-aware models, validator_list or validate()?

2006-06-13 Thread gabor

(sorry for the double-post. i only later realized that replying to a 
previous thread is not a good idea)

hi,

i tried to implement the validation-logic for some of the fields, but
while looking at the source code, i found this inconsistency...

question:


class SlugField(Field):
 def __init__(self, *args, **kwargs):
 kwargs['maxlength'] = kwargs.get('maxlength', 50)
 kwargs.setdefault('validator_list', []).append(validators.isSlug)
 # Set db_index=True unless it's been set manually.
 if not kwargs.has_key('db_index'):
 kwargs['db_index'] = True
 Field.__init__(self, *args, **kwargs)
===

as you see, it sets up the validators.isSlug validator

===
class EmailField(CharField):
 def __init__(self, *args, **kwargs):
 kwargs['maxlength'] = 75
 CharField.__init__(self, *args, **kwargs)

 def validate(self, field_data, all_data):
 validators.isValidEmail(field_data, all_data)


this one does not set up a validator, and does a "manual" validation.


why do they behave differently?

or, better question:

what's the recommended approach:
add validators to the field's validator_list, or remove those and do an
explicit validation in the validate() method?

gabor


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: validation-aware models... plans/future?

2006-06-13 Thread gabor

Adrian Holovaty wrote:
> On 6/7/06, gabor <[EMAIL PROTECTED]> wrote:
>> so now the decisions are already done, and when all the fields will have
>> the necessare to_python (+validate) methods, then save will be changed
>> to call validate, and the task is done?
>>
>> or still some decisions have to be done first?
>>
>> i'm simply asking whether patches should be sent in for the remaining
>> fields?
> 
> Right -- those are the next steps. Thanks for the prodding about this.
> After that, the plan is to remove the automatic manipulators in favor
> of something that uses the validation. See Joseph K.'s e-mail to
> django-developers from a couple of weeks/months ago...
> 

i tried to implement the validation-logic for some of the fields, but 
while looking at the source code, i found this inconsistency...

question:


class SlugField(Field):
 def __init__(self, *args, **kwargs):
 kwargs['maxlength'] = kwargs.get('maxlength', 50)
 kwargs.setdefault('validator_list', []).append(validators.isSlug)
 # Set db_index=True unless it's been set manually.
 if not kwargs.has_key('db_index'):
 kwargs['db_index'] = True
 Field.__init__(self, *args, **kwargs)
===

as you see, it sets up the validators.isSlug validator

===
class EmailField(CharField):
 def __init__(self, *args, **kwargs):
 kwargs['maxlength'] = 75
 CharField.__init__(self, *args, **kwargs)

 def validate(self, field_data, all_data):
 validators.isValidEmail(field_data, all_data)


this one does not set up a validator, and does a "manual" validation.


why do they behave differently?

or, better question:

what's the recommended approach:
add validators to the field's validator_list, or remove those and do an 
explicit validation in the validate() method?

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



validation-aware models... plans/future?

2006-06-07 Thread gabor

hi,

after i spent some painful time with the AddManipulator in a non-web 
python script :-), i went to check the status of the validation-aware 
models.

seems the last checkin was http://code.djangoproject.com/changeset/2518.

so now the decisions are already done, and when all the fields will have 
the necessare to_python (+validate) methods, then save will be changed 
to call validate, and the task is done?

or still some decisions have to be done first?

i'm simply asking whether patches should be sent in for the remaining 
fields?

thanks,
gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Site object to contain the protocol? (https..)

2006-06-05 Thread gabor

Adrian Holovaty wrote:
> On 6/5/06, gabor <[EMAIL PROTECTED]> wrote:
>> i'm building a https site, and have many times problems with the Site
>> objects, which kind-of assume that the site is http.
>>
>> would it be a good idea for the site objects to also contain the
>> protocol (http or https)? what do you think?
> 
> I don't think the Site object itself should contain the protocol, but
> I suspect we could solve your problem by making some of the Django
> helper functions more sensitive to http/https differences. There are
> probably several places within Django in which the "http://; is
> hardcoded.

yes, a quick "grep http:// * -r" shows many places :)

so, would you prefer to fix them one-by-one with something like:

-if url.startswith('http://')
+if url.startswith('http://') or url.startswith('https://')
?

but, i'm afraid this issue is not that simple. it's not just the "if"s.
add_domain from django.contrib.syndication.feeds.py is a good example:

def add_domain(domain, url):
 if not url.startswith('http://'):
 url = u'http://%s%s' % (domain, url)
 return url

how should this be properly "fixed"? :-(


most of these problematic places follow the:

"if url is a full url, use it as it is, otherwise change it into a full 
url by adding the domain name to it"

pattern, so maybe django could have a helper function for this case, and 
all django code could use it...

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Site object to contain the protocol? (https..)

2006-06-05 Thread gabor

hi,

i'm building a https site, and have many times problems with the Site 
objects, which kind-of assume that the site is http.

would it be a good idea for the site objects to also contain the 
protocol (http or https)? what do you think?


p.s: yes, i already mentioned it in my previous post 
(http://tinyurl.com/qm6ny), but i thought that maybe it would make sense 
to mention it (also) in a separate mail.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: get_next_by_FIELD taking parameters (WAS: Re: interesting stuff in django.db.models.base.Model._get_next_or_previous_in_order)

2006-06-01 Thread gabor

Adrian Holovaty wrote:
> On 6/1/06, gabor <[EMAIL PROTECTED]> wrote:
>> any chance of this patch getting applied?
>> http://code.djangoproject.com/ticket/1857
>>
>> it fixes the problem (bug?) that get_next_by_FIELD and
>> get_previous_by_FIELD don't take params.
> 
> Hmmm, I'm not sure they should take parameters, as it doesn't fit well
> with the magic-removal style...What do people think?
> 

thanks for dealing with this ticket... some comments:

1. if they should not, then the documentation will have to be corrected 
(http://tinyurl.com/rh97z)

2. what magic-removal style alternative can be used? for example,let's 
say i have a model with a  pub_date, and a color field. now i have an 
instance of this model, and i need the next (by pub_date) yellow 
instance. how can i query for that one?

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: unicode.. reject?

2006-05-30 Thread gabor

Ivan Sagalaev wrote:
> gabor wrote:
> 
>> i'm also willing to help with this task... but..do i understand this 
>> correctly, that it's agreed that django is going to switch to unicode?
>>  
>>
> Being one of the proponent of the all-unicode way back when it was 
> proposed I should say that the more I think of it the more I'm afraid 
> that it can create just as much problems as it will solve. 

hi,

a short summary at the beginning:
(to borrow the words of someone from sci.lang.japan):

This is a very hard subject to discuss without creating a personal blog,
by the way.

:))

i understand your point, but i still think it pays off to switch to 
unicode. but it needs a confirmation from the django-BDFL's side that 
they are willing to make this step. if not, there's no point to work on it.

generally i think, that you must understand the various aspects of 
unicode. whether you use unicode-strings or byte-string, you have to 
understand them.

below are more various rants regarding the topic :) (if it sounds 
too agressive, then sorry, wasn't my intent)

> Today there 
> is a convention that Django works everywhere with byte strings except 
> some rare cases where it is convinient to use Python's built-in unicode 
> functions (these cases include counting length in validation code and 
> various string conversions in filters like in the unapplied patch in 
> ticket http://code.djangoproject.com/ticket/924). Using this convention 
> one can write international apps without worries since all messy things 
> are made inside the framework.

well, i would replace 'without worries' with 'good enough' :)

> 
> If we change this convention to using Python's unicode everywhere we 
> will hardly win anything except some feeling of a more "pure" approach. 
> At least _I_ can't see any gain :-).

some people would say (maybe including me) that it's already a gain :)

> However there will be some 
> disadvantages that I think are serious:
> 
> - I heard that many 3rd party libraries don't work with unicode so a 
> user will be forced to do some coversions manually

this happens both ways. some libraries work with unicode strings, so you 
have to convert back there :)

for example, i was building a very simple web-file-manager. you can ask 
python to get you the os.listdir, etc, data in unicode, which i think 
the only sensible way, because otherwise you have to watch out for the 
filesystem-encoding. but because the querystring and httpresponse is 
bytestring, i have .encode('utf8'), .decode('utf8') all the way.

> - byte strings is a default string type in Python and is more widely 
> used than unicode, I'm afraid many people in ASCII-world will be 
> resistant to switching all their coding to unicode because they can't be 
> sure that it works right because they can't test it easily themselves

hmm... is it that hard to get some japanese text and maybe some german 
names? :)

anyway, i think it actually might help them at the end, because 
converting your app to unicode means to think about all the places where 
data enters the system and where data leaves the system.

so this is more explicit, and, as 'import this' says: explicit is 
bettern than implicit :))

> 
> So may be Django should just wait for Python 3000 when unicode string 
> will become default ones and developers would be more widely aware of 
> this change.

'aware of this change' is an understatement :) there will not be any 
byte-strings in py3000 :)


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: unicode.. reject?

2006-05-29 Thread gabor

Jeroen Ruigrok van der Werven wrote:
> On 5/29/06, gabor <[EMAIL PROTECTED]> wrote:>
>> but then, to have a nice unicode-integration in django, we'd
>> need to convert EVERYTHING to unicode..httpRequest, httpResponse etc.
>> (as at http://code.djangoproject.com/wiki/UnicodeInDjango).
> 
> 
> As far as I know it that page is the official stance of Unicode in
> Django. Georg (hugo) is the prime instigator and I will help out on
> this since I will probably need some of this functionality.
> 

i'm also willing to help with this task... but..do i understand this 
correctly, that it's agreed that django is going to switch to unicode?

because i don't really see a way to have it coexist with the current 
bytestring-based framework.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: unicode.. reject?

2006-05-29 Thread gabor

Jeroen Ruigrok van der Werven wrote:
> On 5/29/06, gabor <[EMAIL PROTECTED]> wrote:
>> it works fine in all the cases...
> 
> At least that's good news. I just wonder if it means that psycopg is a
> dead-end for dealing with Unicode and psycopg2 should be the preferred
> choice. I will try to see what I can dig up.
> 

hmm, it depends on what do you call "unicode" :)

for example,if you keep EVERYTHING in utf-8 encoded bytestrings,
and:
- generate utf-8 webpages
- set up the database to store data in utf-8,

then you can easily store-and-handle any kind of text (japanese, arabic, 
whatever) in your application even with psycopg1 fine.

but by "unicode" you mean python unicode-strings,hmm..then probably 
psycopg2. but then, to have a nice unicode-integration in django, we'd 
need to convert EVERYTHING to unicode..httpRequest, httpResponse etc. 
(as at http://code.djangoproject.com/wiki/UnicodeInDjango).

i'd love to have django in unicode, but it's not an easy change. and 
also i'm not sure what would the authors say :)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: unicode.. reject?

2006-05-29 Thread gabor

Jeroen Ruigrok van der Werven wrote:
> On 5/28/06, gabor <[EMAIL PROTECTED]> wrote:
>> further debugging showed, that psycopg is at fault, because it quotes
>> byte-string params, but not unicode-string params.
>>
>> also, take an unicode string, like u"gábor" (my name :-). you can assign
>> it into a charfield, and if you save the model using sqlite3, it's going
>> to be ok. but if you do it with psycopg, you get a segmentation fault
>> (on fedora), or a bus-error (on osx). this is again a problem with psycopg.
> 
> Comment from out of nowhere: how does psycopg2 hold itself in these cases?
> 

it works fine in all the cases...

now i just have to install mysql and check that out too :)


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: __getitem__ ignores Meta_ordering?

2006-05-23 Thread gabor

Gábor Farkas wrote:
> Gábor Farkas wrote:
>> hi,
>>
>> is it intentional that calling __getitem__ on a queryset ignores the 
>> Meta_ordering?
>>
>> for example:
>>
>> 
>> class Thing(Model):
>>  name = SlugField()
>>  pub_date = DateTimeField()
>>
>>  class Meta:
>>  ordering = ['-pub_date']
>> 
>>
>>  >>> Thing.objects.all()
>>
>> 'SELECT  FROM "main_thing" ORDER BY "main_thing"."pub_date" DESC'
>>
>> this orders correctly
>>
>>
>>
>>  >>> Thing.objects.all()[0]
>>
>> SELECT  FROM "main_thing" LIMIT 1
>>
>> this does not.
> 
> some more info... it seems the problems is with the following code:

(sorry, i changed the subject-line + text, when i found out that used 
the wrong term) (slicing => __getitem__ )

after some more research, i arrived here:
> 
> ==QuerySet
>  def get(self, *args, **kwargs):
>  clone = self.filter(*args, **kwargs)
>  if not clone._order_by:
>  clone._order_by = ()
> ==

it was added in
http://code.djangoproject.com/changeset/2393,

which was a merge from
http://code.djangoproject.com/changeset/2392,

which is based on the following discussion:

http://tinyurl.com/sxlcp

so it was a performance (cleanness) optimalization for queryset.get().

it makes sense in the get() case, because get() can be only used if the 
queryset only contains one item.

but the problem is that __getitem__ calls get().


a quick hack, like: http://django.pastebin.com/733366 helps,
and all tests pass.

but i'm not sure if this is the nicest way to solve this problem.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: reverse-lookups, strange behaviour

2006-05-22 Thread gabor

Jorge Gajon wrote:
> 
> On 5/19/06, Gábor Farkas <[EMAIL PROTECTED]> wrote:
>> and the strange thing (the reason why i'm reporting this) is that just
>> based on the difference in the "from" statement, i get different results
>> (works in first case, fails in second case).
>>
> 
> You should always use 'from' like this:
> from proj1.app1.models import MyModel

i understand that it's safer to do, but quite ugly in my opinion.
why should my code depend on the name of the project, after all..

> 
> Also, I'm recalling from memory so I could be totally wrong here, but
> I kind of remember that you would fetch the related set


yes, related_name works like that. but it works (and fails) the same way 
as the "unnamed" reverse-lookup.

> I normally look into the tests that come in trunk/tests first. They
> are very good to find out how to use some feature.
> 

actually, that was the place where i started (when i saw that my reverse 
lookups do not work), and found that the test's reverse lookups do work.

and for some strange reason, this only happens in non-web (manage.py 
shell, or "standalone" python script) situations. in "normal" view code 
i can use this import without problems.

anyway, it's not such a big problem as such. the problem i think is that 
it can lead to confusing situations (the "non-reverse" lookups work fine 
  in the from-app1.models situation too).


thanks,
gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



reverse-lookups, strange behaviour

2006-05-17 Thread gabor

hi,

i had a problem with reverse-lookups, and found this strange anomaly:

===
class User(models.Model):
 name = models.CharField(maxlength=200)

class Poll(models.Model):
 question = models.CharField(maxlength=200)
 creator = models.ForeignKey(User)
===

project name : proj1
app name : app1

this works:

===
from proj1.app1.models import User
User.objects.get(poll__question__exact='x')
===

this fails:

===
from app1.models import User
User.objects.get(poll__question__exact='x')
===

(fails with : "TypeError: Cannot resolve keyword 'poll' into field", 
full stacktrace at http://tinyurl.com/jyp3m)

but this problem only happens with reverse lookups. "normal" lookups 
work fine in both cases.

any ideas why this is happening?

i'm not sure if i'm doing something stupid here, or this is a bug.

thanks,
gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: SlugField utf-8 support

2006-04-14 Thread gabor

Ivan Sagalaev wrote:
> And in 
> case of Firefox am I supposed to stick a big red banner on start page 
> saying "To see this page properly in your browser go to 
> about:config..."? :-)
> 
as i understand, that's not required. firefox will correctly go to the 
required url. it just will display it url-encoded...

example

http://www.example.com/gábor.

every browser should-submit/submits the request to the webserver as:

http://www.example.com/g%C3%A1bor

but the difference is that the location-bar in safari will show
http://www.example.com/gábor

and in firefox will show

http://www.example.com/g%C3%A1bor

(unless you switch that setting)


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: SlugField utf-8 support

2006-04-13 Thread gabor

Viktor wrote:
> The problem begins when you want to have nonenglish slug (slug on a 
> language that uses nonascii characters).
> For example if I want to have a serban cyrillic slug: наслов_на_српском, 
> it is perfectly valid slug if you read it on serbian, just aphanumeric 
> characters + underscore + hyphen, nothing more... But the current isSlug 
> validator does not accept it.

i see...

seems that it boils down to what you call 'valid'...
i had the feeling that slugs were meant to be used as parts of URLs...
and i'm sure you know that those cyrillic slugs, if part of an url will 
get submitted in URLencoded form. even if it is hidden by the browser.


anyway...maybe it would be a good idea to add a boolean setting that 
limits the slug to ascii (the old behaviour) ... like

slug = SlugField(only_ascii=True),

or something like that, because i can imagine that some people want to 
restrict to that.


p.s: to make sure we understand each other. i understand the need for 
being able to create non-english links/urls. i'm just not sure if a Slug 
is the best place for it.

p.s.2: btw. are non-english URLs common/widespread nowadays?

gabor

> 
> gabor wrote:
>   > could you please describe some use-cases for this feature?
>> because for me it seems a little...strange..
>> the whole point of the Slug is to only contain alphanumeric (as in 
>> ascii-alphanumeric) characters + underscore + hyphen.
>>
>> with the utf-8 change it can contain underscore+hyphen + all unicode 
>> symbols that are considered as alphanumeric...
>>
>> but if you allow so many symbols, why to use a SlugField at all? why 
>> don't allow all the unicode characters?
>>
>> i'm pretty sure i'm missing something there... :-(
> 
> 
> 
> > 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: SlugField utf-8 support

2006-04-13 Thread gabor

Viktor wrote:
> The story about non ASCII characters in slug started in #1602: urlify.js 
> blocks out non-English chars.
> 
> We tested utf-8 urls against ie, opera and konqueror and they work 
> great. Firefox by default shows the urls urlencoded, but you can change:
>   network.standard-url.escape-utf8 to false
> and it will show the url in original encoding.
> 
> 
> So I changed the validator for the SlugField so it can accept utf-8 strings.
> 

could you please describe some use-cases for this feature?

because for me it seems a little...strange..
the whole point of the Slug is to only contain alphanumeric (as in 
ascii-alphanumeric) characters + underscore + hyphen.

with the utf-8 change it can contain underscore+hyphen + all unicode 
symbols that are considered as alphanumeric...

but if you allow so many symbols, why to use a SlugField at all? why 
don't allow all the unicode characters?

i'm pretty sure i'm missing something there... :-(


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Connections not being closed in m-r

2006-04-05 Thread gabor

Jacob Kaplan-Moss wrote:
> 
> On Apr 5, 2006, at 2:09 PM, Ivan Sagalaev wrote:
>> How can this work? Django app connects to Postgres through psycopg,  
>> how
>> Apache can "hold" the connections that it doesn't know anything about?
>> Or does it?
> 
> It's not "holding" connections, per se... let me see if I can explain  
> it more clearly:
> 
> So the "problem" with Python (when it comes to web apps) is that  
> starting the interpreter is very expensive; on my main web server  
> ``time python -c 'import django.models'`` takes 1.5 seconds!  (This  
> is why anyone trying to do Django as a CGI is crazy).
> 
> To make this a non-issue, any sane environment shares one or more  
> Python interpreters across multiple requests (mod_python has one  
> interpreter per Apache child).  This means that any global variables  
> -- including the all-important ``sys.modules`` -- stay resident in  
> memory between requests.  Since Django's database connection is an  
> attribute in ``django.db``, it also sticks around between requests,  
> hence Postgres reporting idle connections.
> 
> Does that make sense?
> 
> I've consulted a database guru (Frank Wiles, http://revsys.com/)  
> about these "idle in transaction" connections a few times, and he  
> reassures me that they are less of a problem than recreating a  
> database connection upon each request.  His suggestion -- and mine --  
> is that if it's causing issues you should install a connection pool  
> (pg_pool is the best one) between Django and your database.
> 
> Again, I really want to back this change out.  My off-the-cuff  
> measurements (not very good, probably) show about a 5% performance  
> hit when closing the connection each time, and although that's not  
> much, that's 5% less hardware I have to buy.
> 
> I'm sorry I didn't speak up *before* this change got checked in; I  
> missed the thread until I saw the changeset.  Are there any serious  
> objections to backing this out?


hi Jacob, many thanks for the explanation.

(and sorry to jump into the discussion)..

the original poster (luke plant) said that:

"
It appears
that connections are not being closed at all.  With every hit I'm
getting another entry in the process table
"

so if he's only hit by the reuse-open-connection effect, it should be 
pretty easy to reproduce:

setup apache with maxclients = 3, and do more than 3 requests.

if the number of open psql connections stays 3, everything is ok.
if it increases, there's a problem.

perhaps this would help to clarify the situation.


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: magic-removal: plans/estimates for the trunk-merge?

2006-03-24 Thread gabor

ChaosKCW wrote:
> Hi
> 
> I am very interested in starting aproject at my company in Django, but
> am not keen on starting one of 0.91. So I am hoping the MR will be
> merged soon too :-)
> 
> 

actually, i'm in a similar situation.

we have already developed something, which is for now quite simple (6-7 
models, functionality is mostly only the default admin-stuff provided by 
django), which will be probably released next month. so i am waiting if 
i'll be able to change it to magic-removal :)

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



magic-removal: plans/estimates for the trunk-merge?

2006-03-22 Thread gabor

hi,

is there a plan or a rough estimate about when the magic-removal branch 
is going to be merged back to the trunk?

when finished?
this month?
next year?
asap?

and is there anything mere mortals (read: not django-internals experts) 
can do/help-with to make it happen (sooner) :-)?

thanks,
gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Multiple Authentication Backends

2006-03-14 Thread gabor

Max Battcher wrote:
> Authentication/Authorization really doesn't belong in every view.  That 
> way leads to bad coupling and lots of repetition.

please note, that even with the current auth-system, you have to handle 
it in every view.

gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: Automatically generate input type="password" for security-sensitive fields?

2006-03-12 Thread gabor

James Bennett wrote:
> Someone in the IRC channel tonight was asking why the built-in User
> model, when rendered by formfields, doesn't render the password field
> as an input of type "password", and it struck me that this is a good
> question, because there are plenty of use cases where you'd want that
> behavior to happen.
> 
> The question, then, is: what would be a good way to handle this in
> Django? There are three possible responses that occur to me:
> 
> 1. Allow an option in model definitions to indicate that a field is
> security-sensitive; e.g.: password = models.CharField(secure=True).
> 2. Allow an option, similar to the above, to be specified in the
> creation of custom manipulators.
> 3. Do nothing in Django itself, and advise people to just hard-code
> the appropriate input type into their templates.


hmm.. what about this:


class Passwordtest(Model):
password = meta.PasswordField(hashfn = my_password_hasher)

def my_password_hasher(password):
return


this would work the following way:

1. when it is saved (somewhere around pre_save), the hash-value would be 
calculated, and that would be stored in the db
2. the admin-interface would generate a standard type-password-twice 
style gui


i understand that #1 can be achieved with a normal pre-save... but the 
same can be said about many of the fields (urlfield, emailfield and so 
on). i think that passwords are common enough that a field dedicated for 
them would be necessary.



gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: templates and html escaping

2006-03-03 Thread gabor

Max Battcher wrote:
> On 3/3/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>> Jacob Kaplan-Moss wrote:
>>
>>> Why not do it for all variables? At times you want to pass chunks of
>>> HTML into a template that get displayed raw.  I don't think the
>>> behavior you suggest should be default, but do you have any ideas on
>>> how to make it optional?
>> the standard solution is to use a wrapper for strings that you
>> *don't* want escaped.  see e.g. the section on "HTML templates"
>> on this page:
> 
> The other problem is that not everyone wants the same escape.  For
> instance, there are those of us that {% load markup %} and then
> |restructuredtext (or |markdown or |textile) just about everything,
> and auto-HTML escape could certainly cause havoc here.

those filters could first de-html-escape the content.

i understand that it seems silly for this configuration. i only wanted 
to demonstrate that even with auto-html-escaping you can still use those 
markups fine.


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



Re: urlconf, applications..coupling

2006-02-26 Thread gabor

Max Battcher wrote:
> gabor wrote:
>> i actually got around this problem by adding the project-dir to the 
>> PYTHONPATH, but it seems a little as a hack...
> 
> I'd have to disagree.  I don't think it's a hack at all, as far as I'm 
> concerned it makes sense if you think of the project directory as a 
> "library namespace" 

i see...
my problem is then maybe... a phylosophical one..
how to explain..

i have a bad feeling when i'm doing something differently than the 
official tutorial :)
it makes me feel that i'm doing it not the django way :)


or to formulate it differently:

why is the tutorial doing it the way it does?
wouldn't it make more sense to do it the project-dir-on-pythonpath way?

> (you don't expect the django framework itself, or 
> any other Python library, to drop their namespace when you start using 
> them in your own project, do you?).
> 

could you explain this in a little more detail?
(it's 00:26 already here, so i'm not the brightest right now ;)

thanks,
gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---



urlconf, applications..coupling

2006-02-26 Thread gabor

hi,

http://www.djangoproject.com/documentation/tutorial3/

the "include" functionality in urlconfs is demonstrated,because it 
allows you to decouple your application's urlconfig from the projects 
urlconfig.

but the problem is,that you still have to use "absolute" imports in the 
application's urlconf.

when talking about the example in the tutorial,

the urlconf for the polls app is:

urlpatterns = patterns('myproject.polls.views',
 (r'^$', 'index'),
 (r'^(?P\d+)/$', 'detail'),
 (r'^(?P\d+)/results/$', 'results'),
 (r'^(?P\d+)/vote/$', 'vote'),
)

it is nice that the "URL" handling is decupled, so  we only deal with 
the part of the URL that is important for us.

but we still have to specify the whole name for the module 
"myproject.polls.views". so if later i want to use this application in a 
different project, i will have to change the urlconf in the application.
wouldn't it be better if i could simply say "polls.views.detail", or 
"views.detail" ?


are there any other ways to do this?
or maybe i got this completely wrong :) ?

i actually got around this problem by adding the project-dir to the 
PYTHONPATH, but it seems a little as a hack...


gabor

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---