Re: Proposal: The 'use' template tag, a cross between 'include' and 'extends'.

[Ian Kelly]

On Wed, Sep 3, 2014 at 10:10 AM, Sam Willis  wrote:
> Although I have implemented this with the 'use' word, there may be a better
> word. I considered 'embed' but thought 'use' was a little cleaner

Since it's so similar to 'include', is there a reason not to just add
the new functionality to the existing tag?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALwzidndLpSLYne8egrWZ4qQwXkTUjn-ZB%3DHjPVQj%3DP1gLa1Qw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.7 release status (RC2 coming soon)

[Ian Kelly]

On Fri, Jul 25, 2014 at 7:12 AM, Ryan Hiebert  wrote:
>
>> On Jul 25, 2014, at 12:22 AM, Ian Kelly  wrote:
>>
>> It seems to me that a new release would be useful to have for the
>> reasons given, but it should be called what it is: a beta version, not
>> an RC.
>>
> I agree, but I don’t think that releasing a beta after an RC makes good sense 
> either. If there was a really significant benefit to the release, then 
> calling it an RC even though it’s more of a beta seems like a reasonable 
> compromise. I don’t really see a significant benefit to the release over just 
> using latest git branch, though.

I see two benefits. One is that users who are interested in testing
aren't necessarily going to think to go to github; they're likely to
look for the most recent release and use that (on the other hand, if
they see a beta 4 and an RC 1, they might not necessarily realize the
beta 4 is most recent either, so I take your point). The second
benefit is that the instructions "pull the latest from github" can
*sound* like a lot of work if the user is not a contributor and isn't
experienced with git, and that might deter them from testing.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALwzidmuX7HOFOz5eF7RGiH97Ny%3D0wO6C9UO1XMFy5%3DA7A0p8Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.7 release status (RC2 coming soon)

[Ian Kelly]

On Thu, Jul 24, 2014 at 5:49 PM, Ryan Hiebert  wrote:
> My opinion is worth less than 2c, but I’m inclined to agree with the dissent.
>
> It seems to me that its easy enough to install the very latest from the 
> github repository versus from a tarball, so leaving RC versions to actually 
> be candidates for release seems like a reasonable idea. Were we releasing 
> prerelease versions on PyPI, then I’d probably reverse that opinion, because 
> at that point it would be easier to install (though only marginally).

It seems to me that a new release would be useful to have for the
reasons given, but it should be called what it is: a beta version, not
an RC.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALwzidk74Xe9LZ%2BYmDvgJrs6NNcBdxxFK63jZ6h8Y%2BVC%3DYhO9w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Supporting and using EOL'd software (was Re: Time to drop support for Oracle < 11?)

[Ian Kelly]

On Sun, Jun 15, 2014 at 8:59 AM, Michael Manfre  wrote:
> I don't see how it should be up to Django to continue to support all of
> these archaic versions of Oracle. To paraphrase the mantra repeated during
> various mssql discussions, "Django doesn't need to include everything in
> core, it just needs to make it possible for others to implement". If people
> feel strongly about staying on a no longer supported version of Oracle, then
> they can either stick with an older version of Django or create
> django-decrepit-oracle.
>
> Django should strip this legacy code.

There's not a lot of that actually. Dropping 9i would let us get rid
of the regex version check:

https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L361
https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L648

and this one-line database ping workaround:

https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L715

and I'm not coming up with anything else.

Dropping 10g as well wouldn't definitely let us remove anything. It
might make it easy to fix #11580 for doing icontains queries on
TextFields, and it *might* let us get rid of the complicated
LIKE/LIKEC fallback from #14149.  Both of these would need further
research and testing.  Particularly in the LIKE/LIKEC case, as that
bug has a long and checkered past, with a history of failures on
*some* (apparently non-continuous) versions of Oracle 9 and *some*
versions of Oracle 10, and it may also be sensitive to database
configuration, which makes it hard to be sure whether it's finally
fixed in Oracle 11 or not.

Are there any other places where Oracle 9/10 may be holding back the
code base that I'm not aware of?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALwzidkSZ%3DsVNDST%3Dpv0VOaTxvSVHZBkaCY%3D-H-SzJzQo3Xfww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Time to drop support for Oracle < 11?

[Ian Kelly]

On Fri, Jun 13, 2014 at 7:40 PM, Shai Berger  wrote:
> Hi Tim and all,
>
> On Saturday 14 June 2014 03:52:57 Tim Graham wrote:
>>
>> Release - GA Date - Premier Support Ends - Extended Support Ends
>>
>> 11.1 - Aug 2007 - Aug 2012 - Aug 2015
>> 10.2 - Jul 2005 - Jul 2010 - Jul 2013
>> 10.1 Jan 2004 - Jan 2009 - Jan 2012
>> 9.2 Jul 2002 - Jul 2007 - Jul 2010
>>
>> According to our docs
>> :
>> Django supports Oracle Database Server  versions 9i
>> and higher. Oracle version 10g or later is required to use Django's regex
>> and iregex query operators.
>>
>> Is it time to drop support for versions of Oracle that are no longer
>> supported?
>
> Heh. I tried this one myself, 2.5 years ago:
> https://groups.google.com/d/msg/django-developers/SQ0Pltt_f0M/-h0jvJc5p1cJ
>
> Unlike ikelly at that time, I'm not aware of people still using Oracle 8 or 9,
> but reports about Oracle 10 were still received not too long ago (e.g.
> https://code.djangoproject.com/ticket/20292 includes a report by an Oracle 10
> user from a year ago). I don't want to pull rugs from under these people's
> feet, if they are still out there.

I'm pretty much out of the Oracle fold at this point, so I don't know
either how common these installations still are.  A stackoverflow
search for "[oracle9i]" shows that several new questions per month are
still being asked there. "[oracle9i] [django]" is looking pretty
empty, though.

My only comment at this point is that if support is to be dropped,
#11580 doesn't make a particularly strong case for it since running an
icontains query on a TextField isn't going to scale well and probably
isn't a good idea to start with.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALwzid%3DWHHHqe-a0UcR1EX8zZx4VtXicPFjNHVL91AHqGXUyJw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Support Negative Indexing on QuerySets

[Ian Kelly]

On Tue, Jul 30, 2013 at 4:04 PM, Wim Lewis  wrote:
>
> On 30 Jul 2013, at 2:06 PM, Florian Apolloner wrote:
>> How do you think such support would look like? For negative indices you'd 
>> have to know the size of the resultset to be able to do "limit somthing 
>> offset length-your_negative_index" -- this doesn't seem to make any sense 
>> for an ORM. You can always do list(qs)]:-1] though…
>
> It seems like the first comment in the ticket answers that question. Django 
> would reverse the sense of the query's ordering clause and use a simple LIMIT.

This would only work if the ordering specified were total.  If it were
not a total ordering, then I believe there is no guarantee that the
DESC order will simply be the reverse of the ASC order.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Re: History of datetime_cast_sql() in Oracle

[Ian Kelly]

On Sun, May 19, 2013 at 7:33 AM, Shai Berger  wrote:
> Hi,
>
> In my quest to solve Oracle bugs, I found the function datetime_cast_sql() in
> the Oracle backend. It causes a problem these days, because it gets in the way
> of querying a date as string (mydate__startswith='2005'). It is a function
> that is defined as a no-op in the base backend, and overridden, among core
> backends, only in Oracle. Removing the override, currently, just makes one
> more test pass, and if we decide to remove it from the base, code in several
> other places may become simpler (will not need to special-case datetime
> values).
>
> I was able to track the history of this function, into the mists of time. In
> the beginning, code to cast strings into dates for Oracle was mixed in with
> the general querying code. Then came the boulder-oracle-sprint of 2006-2007,
> where, at some point, that piece of code was factored out to a module-level
> function get_datetime_cast_sql(), which was defined in all backends as a no-op
> except Oracle; this was later refactored into the DatabaseOperations class we
> know today.
>
> However, at about the same time that the DatabaseOperations was created, the
> Oracle backend made another change: It started setting the session's datetime
> format on login. This, as far as I understand, makes the special casting
> operation redundant -- and current test results support my understanding.
>
> So -- I want to fix, now, the thing that was, well, not broken, but bent, in
> 2007. And my question to you -- especially, those of you who participated in
> the boulder sprint -- can you think of any reason why I shouldn't?

You may be right.  One thing I would be concerned about is the
reliability of only doing implicit date conversions.  If you google
for "ORA-01843" you can find a good number of forum posts that
recommend against relying on the NLS_TIMESTAMP_FORMAT setting.  See
also Django ticket #20292.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: URL dispatcher fallthrough?

[Ian Kelly]

On Tue, Mar 19, 2013 at 12:26 PM, Felipe Prenholato
 wrote:
> Hi Julian. My 2 cents.
>
> What you want is:
>
> urlpatterns = patterns('',
> url_fallthrought(r'^(?P[-\w]+)$',
> views=[(ProductView.as_view(),
>CategorView.as_view(),
>OccasionView.as_view())], name="my_super_url"),)

def view_chain(views):
def inner(*args, **kwargs):
for view in views:
try:
return view(*args, **kwargs)
except Http404:
pass
raise Http404
return inner

urlpatterns = patterns('',
(r'^(?P[-\w]+)$',
 view_chain([ProductView.as_view(),
 CategoryView.as_view(),
 OccasionView.as_view()]), name="my_super_url"),
)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 3:03 PM, Petite Abeille
 wrote:
> Another point perhaps worthwhile mentioning:
>
> # There's no way for the DatabaseOperations class to know the
> # currently active Oracle version, so we do some setups here.
>
> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L574
>
> That information can be extracted from v$version, v$instance, 
> dbms_db_version.version, product_component_version, etc, etc. The choice is 
> yours.

This is actually a timing concern.  The DatabaseOperations class can't
determine the Oracle version on its own because it doesn't have a
connection instance.  So instead the DatabaseWrapper selects the
appropriate regex method for the Oracle version when the connection is
created.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 2:15 PM, Petite Abeille
 wrote:
>
> On Mar 10, 2013, at 8:28 PM, Aymeric Augustin 
>  wrote:
>
>> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L548-L555
>
> Last but not least… I couldn't help notice these suspicious looking operators:
>
> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L479
> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L495
>
> E.g. TRANSLATE … USING NCHAR_CS… and LIKEC…
>
> I suspect these are some sort of workarounds some fundamental charset 
> encoding misunderstandings :)
>
> Perhaps best to revisit that in light of "Supporting Multilingual Databases 
> with Unicode" and "Programming with Unicode":
>
> http://docs.oracle.com/cd/E14072_01/server.112/e10729/ch6unicode.htm
> http://docs.oracle.com/cd/E14072_01/server.112/e10729/ch7progrunicode.htm

These particular lookups have a long history of being tweaked due to
users coming up with installations where the existing queries did not
work.  See tickets #5985, #11017 and #14149.  I'd rather not reopen
this issue unless the current implementation can documentably be shown
to be broken.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 1:44 PM, Petite Abeille
 wrote:
>
> On Mar 10, 2013, at 8:28 PM, Aymeric Augustin 
>  wrote:
>
>> Django does this already:
>> https://github.com/django/django/blob/master/django/db/backends/oracle/base.py#L548-L555
>
> Perfect.
>
> In the case of #20015, the issue is the other way round… the literal '2008%' 
> cannot be implicitly converted to a date. Implicit conversion always go to 
> the narrowest type. You would be better off converting it explicitly, e.g. 
> to_date( '2008', ''  ) or such. Then the query become date >= to_date( 
> '2008', ''  ). Alternatively, you could convert the date column to a 
> varchar: to_char( date ) like '2008%'…

The lookup requested by the user in this somewhat obscure scenario is
fundamentally a string lookup, though.  For example, this alternate
lookup is entirely legitimate as far as Django is concerned:

MyModel.objects.filter(somedate__startswith='200')

Which could match anything with the year 200 but more likely is
intended as a decade match, matching any date in the 200X decade.  To
support that, the query would have to become something like "somedate
between to_date('2000', '') and to_date('2009-12-31
23:59:59.99', '-MM-DD HH:MM:SS.FF')"

And there would have to be different conversions for other possible
substring lookups.  Generating those kinds of queries from Django
string lookups would be a hairy mess.

Digging into the code a bit further, I see that the postgresql backend
accomplishes this by tacking '::text' onto the end of the field name
when the lookup is a string-based lookup like startswith.  Oracle
could probably do something similar with the TO_CHAR function.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Test failures under Oracle

[Ian Kelly]

On Sun, Mar 10, 2013 at 1:52 AM, Aymeric Augustin
 wrote:
> Hi folks,
>
> At the moment, a few tests are failing under Oracle. I've created tickets for 
> each of them:
>
> https://code.djangoproject.com/ticket/20010
> https://code.djangoproject.com/ticket/20011
> https://code.djangoproject.com/ticket/20012
> https://code.djangoproject.com/ticket/20013
> https://code.djangoproject.com/ticket/20014
> https://code.djangoproject.com/ticket/20015
>
> The core team is notoriously under-qualified when it comes to maintaining 
> support for Oracle — we don't have 6-figure budgets for consulting and 
> Oracle's online docs are a sad joke.
>
> If you'd like future versions of Django to continue supporting Oracle, please 
> have a look at these tickets and help us fix them!

#20015 looks like an expected failure.  Oracle supports lookups of
date fields using strings by implicitly converting the string to a
date.  The test is doing a startswith lookup that is going to produce
sql that looks something like "WHERE date_column LIKE '2008%'".
That's not going to work because Oracle can't convert '2008%' into a
date.  So the test here is testing that the
'supports_date_lookup_using_string' feature is more comprehensive than
Oracle actually supports.

I'll try to make time to look at the other tickets in more depth tomorrow.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Ian Kelly]

On Tue, Mar 5, 2013 at 3:13 PM, Aymeric Augustin
 wrote:
> In the mean time, I discovered that it's impossible to implement
> TransactionMiddleware reliably as a middleware, because there's no guarantee
> that process_response or process_exception will be called.

Perhaps this could be fixed.  We could add another middleware method
called process_finally or somesuch, that would be guaranteed to be
called on every request after the other methods.  It would take a
request and response as arguments, and any return value would be
ignored.  If it raised an exception, the exception would be caught,
logged, and otherwise ignored.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Switch to database-level autocommit

[Ian Kelly]

On Tue, Mar 5, 2013 at 6:38 AM, Shai Berger  wrote:
> I'm not sure what you mean by "unsafe".
>
> Version 1: code in the question
>
> rows = MyModel.objects.all()
> for row in rows:
> try:
> MyModel.objects.get(photo_id=row.photo_id)
> except:
> row.delete()
>
> When Django checks for a second record on get() it reads the second record, so
> a repeatable-read transaction would acquire an exclusive read lock on it. This
> makes it impossible for another transaction to delete the second row before
> the first finishes.

SELECT statements without "FOR UPDATE" do not generally acquire
exclusive locks to my knowledge, even under serializable or repeatable
read isolation levels.  That would be a major issue for parallel
requests if they did.  Also, I don't think there's any distinction
between exclusive read or exclusive write locks; row-level locks are
either just exclusive or shared.

I just verified that, at least in PostgreSQL, the logic above works in
the serializable isolation level but can result in data loss in the
repeatable read isolation level.  I don't have MySQL handy to test,
and Oracle and sqlite3 support serializable and read committed but not
repeatable read. I started two transactions in separate psql shells.
The starting data was:

 id | photo_id
+--
  1 |1
  2 |1

The first transaction ran:

BEGIN;
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;
SELECT * FROM TEST WHERE photo_id = 1;
DELETE FROM TEST WHERE id = 1;
COMMIT;

The second transaction ran the same thing but deleted id 2 instead.
The individual statements were interleaved (so both queries returned
two rows in the SELECT before either query deleted anything).  The end
result was that both rows were deleted.  Trying the same thing in the
serializable isolation level results in this error when attempting to
commit the second transaction:

ERROR:  could not serialize access due to read/write dependencies
among transactions
DETAIL:  Reason code: Canceled on identification as a pivot, during
commit attempt.

> Version 2: code in the first answer.
>
> for row in MyModel.objects.all():
> if MyModel.objects.filter(photo_id=row.photo_id).count() > 1:
> row.delete()
>
> To the best of my understanding, a repeatable-read transaction gets a read
> lock on all records participating in a count, so again, nobody can delete the
> other records before the transaction finishes.

But with any kind of autocommit on, the transaction ends after the
row.delete(), unlocking the remaining rows and allowing them to then
be deleted.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: first() and last(), earliest() and latest()

[Ian Kelly]

On Wed, Feb 27, 2013 at 3:34 PM, Wim Feijen  wrote:
> Which style do you prefer?
>
> .filter(last_name__startswith='b').order_by('last_name').first()# clear
> and long
> .first(last_name__startswith='b').order_by('last_name')# first method
> has filter syntax.
> .filter(last_name__startswith='b').first('last_name')   # first method has
> order_by syntax.

+1 for the first syntax.  The others are duplicating functionality
that is already present via more aptly named methods.  The first
syntax is also more consistent with other ORMs.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: ANNOUNCE: Django 1.5 release candidate 2, Django 1.4.4, Django 1.3.6 (security releases)

[Ian Kelly]

On Feb 20, 2013 4:41 PM, "Carl Meyer"  wrote:
>
> On 02/20/2013 04:25 PM, Nick Phillips wrote:
> >> There was extensive back-and-forth discussion of this in writing the
> >> patch. The issue is that in almost all cases the correct value of the
> >> setting in local development and under test are different from the
> >> correct value in production. So how much value is there in knowing the
> >> tests pass, and/or it works locally? That doesn't really increase the
> >> chances that you have it configured correctly in production.
> >
> > It would at least ensure that you were aware of it and had attempted to
> > set it...
>
> That's true.

If you have a test tier where DEBUG=False, then the first request from an
actual browser would also detect this, would it not?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Request for review / Multiple time zone support for datetime representation

[Ian Kelly]

On Tue, Feb 12, 2013 at 3:35 PM, Aymeric Augustin
 wrote:
>> Against the possible issue of non-recognition of UTC, I suggest using
>> '0:00' instead of 'UTC'.
>
> Thanks for looking into this problem!
>
> I tried '0:00' instead of 'UTC', and '+03:00' instead of
> 'Africa/Nairobi' with the same result. My first queries
> show that conversions are performed correctly
> between these two time zones.
>
> When I pass an invalid time zone name I get:
> ORA-01882: timezone region not found

I didn't mean to suggest this as a fix to the connection dropping
problem.  I only meant that the final SQL should use '0:00' instead of
'UTC' to guard against databases that for whatever reason do not have
a UTC time zone definition, such as the one that I found.  Otherwise
the query would always fail with an ORA-01882 on those databases.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Request for review / Multiple time zone support for datetime representation

[Ian Kelly]

On Tue, Feb 12, 2013 at 3:06 PM, Ian Kelly  wrote:
> On Tue, Feb 12, 2013 at 2:25 PM, Aymeric Augustin
>  wrote:
>>>>> c.execute("""SELECT "DT" FROM "TIMEZONES_EVENT" WHERE EXTRACT(MONTH FROM 
>>>>> CAST((FROM_TZ("DT", 'UTC') AT TIME ZONE (:arg)) AS DATE)) = 1""", 
>>>>> ['Africa/Nairobi']).fetchall()
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> cx_Oracle.OperationalError: ORA-03113: end-of-file on communication channel
>>
>> * That's exactly the same query as above, except the time zone name is 
>> passed in argument.
>> * Not only does it crash, but it closes the database connection!
>>
>> Obviously I'm hitting a bug in Oracle. Does anyone have ideas to solve this?
>
> I encounter the same bug in Oracle 10g XE.  I tried it also on an 11.2
> database, and it seemed to work, but I ran into a different issue:
> neither of the time zones 'Africa/Nairobi' nor 'UTC' existed in the
> database.  I substituted 'Africa/Cairo' and 'Etc/GMT' for the test.
> Wondering if that was just a local configuration issue, I tried it
> again on another newer 11.2 database.  This one did recognize all the
> time zones, but also exhibited the connection loss bug.

Against the possible issue of non-recognition of UTC, I suggest using
'0:00' instead of 'UTC'.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Request for review / Multiple time zone support for datetime representation

[Ian Kelly]

On Tue, Feb 12, 2013 at 2:25 PM, Aymeric Augustin
 wrote:
 c.execute("""SELECT "DT" FROM "TIMEZONES_EVENT" WHERE EXTRACT(MONTH FROM 
 CAST((FROM_TZ("DT", 'UTC') AT TIME ZONE (:arg)) AS DATE)) = 1""", 
 ['Africa/Nairobi']).fetchall()
> Traceback (most recent call last):
>   File "", line 1, in 
> cx_Oracle.OperationalError: ORA-03113: end-of-file on communication channel
>
> * That's exactly the same query as above, except the time zone name is passed 
> in argument.
> * Not only does it crash, but it closes the database connection!
>
> Obviously I'm hitting a bug in Oracle. Does anyone have ideas to solve this?

I encounter the same bug in Oracle 10g XE.  I tried it also on an 11.2
database, and it seemed to work, but I ran into a different issue:
neither of the time zones 'Africa/Nairobi' nor 'UTC' existed in the
database.  I substituted 'Africa/Cairo' and 'Etc/GMT' for the test.
Wondering if that was just a local configuration issue, I tried it
again on another newer 11.2 database.  This one did recognize all the
time zones, but also exhibited the connection loss bug.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Possible ORM bug with SQL syntax & generic relations

[Ian Kelly]

On Wed, Jan 30, 2013 at 10:10 PM, Dylan Klomparens
 wrote:
> It's interesting this seems to work with 10g XE. I am not sure if this is
> relevant, but I am using the Full Oracle Client with Administrative Tools
> (as oppose to the Oracle Instant Client) and cx_Oracle for Python access to
> the database. The error I felt was most problematic is that the SQL
> statement has 4 opening parenthesis, and only 3 closing parenthesis. This
> kind of statement work in Oracle 10g XE?

I noticed that also, but I thought that perhaps it was a transcription
error, since you said it works with sqlite3.  The Oracle version
should not make any difference for this query.  More likely I just
didn't hit the same query.  What version of Django are you using?
Also it might help if you posted your ModelAdmin, in case that's
affecting the query.  My test used a default ModelAdmin.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Possible ORM bug with SQL syntax & generic relations

[Ian Kelly]

On Wed, Jan 30, 2013 at 4:30 PM, Dylan Klomparens
 wrote:
> Hello,
>
> I've encountered what may be a Django ORM bug and am writing to see what the
> dev community thinks. I have a model that uses generic relations. An error
> is returned when I attempt to delete multiple rows of said model in the
> database table. The error is repeatable, and I have found it is only
> applicable to models that contain generic relations. The error occurs after
> the deletion is confirmed in the admin interface and the SQL statement
> begins execution.
>
> I've attached the use case that creates the bug, the generated SQL statement
> (read from the debug output), and the model in question.
>
> Could someone please confirm that this is indeed a bug? If so, I can create
> a ticket and begin researching how to remedy this problem.

I can confirm that the attached SQL is problematic for Oracle.  It has
an ORDER BY clause inside an IN subquery, which is meaningless in
general and not permitted by Oracle.  However, I was not able to
confirm the bug using Django 1.5rc1 and Oracle 10g XE.  I tried
replicating the case with the model provided, and the multiple
deletion was successful.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: URLField definition on creation.py

[Ian Kelly]

On Mon, Dec 17, 2012 at 3:40 PM, maxi  wrote:
> I was seeing that the new URLField [1] is just defined on oracle backend
> (into creation.py)
> Why not in other backend modules?

Actually, I think that line in oracle/creation.py is dead code.  The
internal type of URLField is CharField, so the CharField line would be
used for URLField.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Oracle backend bug makes testing on oracle slower?

[Ian Kelly]

On Tue, Dec 11, 2012 at 6:31 AM, Shai Berger  wrote:
> Hi all,
>
> I've just been hit by a very simple and annoying bug on Oracle, on Django
> 1.3.4. Without testing, it seems to no longer be as bad on master, but a shade
> of it still remains.
>
> The bug is this: The DatabaseFeatures for the Oracle backend do not specify
> that it supports transactions.
>
> On 1.3.4, this means the value is only set if confirm() is called, and when it
> is, it only affects a single connection; it does not appear to be called
> whenever a connection is created -- while database test creation calls it, by
> the time the tests use it it is all forgotten and testing is relegated to the
> old behavior of flush for every test.
>
> On master, where the generic supports_transactions is a cached property, this
> means that for every connection opened, 4 statements are being executed to
> check if the database (still) supports transactions.

While connections come and go, the DatabaseWrapper objects are stored
in the ConnectionHandler on a per-thread basis and should be reused
for subsequent connections.  So these tests *should* run only once per
configured connection per thread.  Can you demonstrate that they are
actually being run on every connection?

> 1. Oracle supports transactions. is there a reason not to just add the line
>
> supports_transactions = True
>
> to its DatabaseFeatures class?

I see no harm in it, although per the above I don't think it will help
as much as you hope.  Since the test involves a CREATE TABLE though,
and since DDL is slow in Oracle, and since there's really no guarantee
that the user even has the necessary permissions to create the table
in the first place, I think it's a good idea to add this in any case.

> 2. Database features, typically, do not change between opening of connections
> within a single server run. Shouldn't we consider saving these features --
> even the dynamic ones -- on the DatabaseFeatures class, instead of its
> instances?

This would lead to bugs in multi-DB setups where the user has
configured connections to multiple databases of the same type but
different versions (and different feature sets).  I don't know how
common that is, but it is better to be correct and slow than fast and
buggy.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Confusion with Unicode

[Ian Kelly]

On Sat, Nov 24, 2012 at 6:55 AM, Santiago Basulto
 wrote:
> Hey guys, i'm posting this here because I posted this on django-users
> yesterday and didn't get any help.
>
> I'm kind of confused here...
>
> If I get data from a request, say:
>
> request.GET.get("something") or request.POST.get("something")
>
> Is that data automatically being encoded based on the Encoding of the
> request? Or I should take care of it explicitly?

The contents of a Django HttpRequest.GET or HttpRequest.POST are
automatically decoded to Unicode for you, using the encoding specified
in HttpRequest.encoding.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django produce sql with table name different then specified in meta db_table

[Ian Kelly]

On Wed, Oct 31, 2012 at 2:57 PM, Łukasz Rekucki  wrote:

> Why all the slashes ? Unless you specify r'' or make them double, they
> don't do anything!
>

Because the OP had them, and for visual clarity, since it can be hard to
distinguish whether '" is an a single quote followed by a double quote or
vice-versa.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django produce sql with table name different then specified in meta db_table

[Ian Kelly]

On Wed, Oct 31, 2012 at 2:18 PM, Michał Nowotka  wrote:

> But what is really buggy here is if i explicitly specify that this
> table is not managed django should respect db_table and not perform
> any truncation. That's all.
>

It won't truncate the name if you fully enclose it in quotes.  Instead of:

db_table = 'schema_name\".\"table_name'

Try:

db_table = '\"schema_name\".\"table_name\"'

This only works for unmanaged models.  For managed models it results in an
error when it tries to create the sequence and trigger using syncdb.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django produce sql with table name different then specified in meta db_table

[Ian Kelly]

On Wed, Oct 31, 2012 at 12:34 PM, Michał Nowotka  wrote:

> Another thing - this name is not truncated - some junk is appended and not
> after 30 characters but earlier and only for some table names regardless of
> length.
>

The truncation actually truncates to 26 characters and then appends the
first four digits of the MD5 hash.  This is done to avoid collisions
between long table names that share a common long prefix.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django produce sql with table name different then specified in meta db_table

[Ian Kelly]

On Wed, Oct 31, 2012 at 1:23 PM, Shai Berger  wrote:

> I'm not sure -- perhaps the db_tablespace option is interpreted by the
> Oracle
> backend as schema, which would allow what Michal is trying to do with a
> sane
> API. Otherwise, I'd advise Michal to look at Oracle's table aliases, and
> mark
> this also as a missing feature.
>

The Oracle backend interprets db_tablespace as a tablespace.  Ultimately,
the proper way to specify a schema will be with the db_schema option, but
this depends on ticket #6148.  Until then, synonyms are the officially
recommended workaround, but the 'schema"."table' hack is also commonly used.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django produce sql with table name different then specified in meta db_table

[Ian Kelly]

On Wed, Oct 31, 2012 at 1:23 PM, Shai Berger  wrote:
> This, almost worthy of being called an sql injection, can't be the right
way
> to achieve the goal. In fact, the Oracle backend (or even some higher,
more
> generic level) should have doubled those '"' characters to make them part
of
> the name. But -- save length issues -- the ploy succeeds:

In fact, Oracle table names are not allowed to contain double quote
characters at all.  Doubling them does not change this.  Per the Oracle
naming rules:

Nonquoted identifiers can contain only alphanumeric characters from your
> database character set and the underscore (_), dollar sign ($), and pound
> sign (#). Database links can also contain periods (.) and "at" signs (@).
> Oracle strongly discourages you from using $ and # in nonquoted identifiers.
>
> Quoted identifiers can contain any characters and punctuations marks as
> well as spaces. *However, neither quoted nor nonquoted identifiers can
> contain double quotation marks* or the null character (\0).
>

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible bug in django oracle backend - no way to map from model type to oracle NUMBER type.

[Ian Kelly]

On Wed, Oct 17, 2012 at 4:26 AM, Michał Nowotka  wrote:
> Currently I'm working with some oracle legacy db which I would like to
> access using django ORM.
> I have a column of type NUMERIC (without precision and scale) and all
> possible values are kept there (for example, 65, 2.3, 0.5)
> After running inspectdb django suggested that this column should be of type
> models.DecimalField(unique= True, null=True, max_digits=0,
> decimal_places=-127, blank=True) but this makes little sense since I'm
> getting an error when trying to do syncdb from this model:
>
> Error: One or more models did not validate: DecimalFields require a
> "decimal_places" attribute that is a non-negative integer.
> DecimalFields require a "max_digits" attribute that is a positive integer.
>
> Do you know what type in model will map to NUMBER in oracle?

There is none.  The proper type would be a floating-point
DecimalField, but Django doesn't allow that as the decimal_places
attribute is required.  I'm not really sure why that is; maybe the
requirement could be relaxed?

If you don't need a lot of precision, then you could probably get away
with using a FloatField.  Otherwise you might want consider creating a
custom field that looks like a FloatField but uses decimals instead of
floats.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Oracle backend vs. numbers

[Ian Kelly]

On Tue, Sep 25, 2012 at 11:47 AM, Ian Kelly  wrote:
> Thanks for putting together a pull request.  Please note the patch
> will also need to update the documentation to state the new cx_Oracle
> version requirement (was 4.3.1, now 5.0.1).

Also, there should probably be something about this in the 1.5 release notes.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Oracle backend vs. numbers

[Ian Kelly]

On Tue, Sep 25, 2012 at 1:31 AM, Shai Berger  wrote:
> On Sunday 23 September 2012, Anssi Kääriäinen wrote:
>> Doing final polish for Ian's patch and providing benchmark results for it
>> will get this patch closer to commit.
>>
> I had hoped the Django test suite alone will provide a convincing benchmark,
> but apparently that is not the case -- on my laptop, with Oracle in a virtual
> machine, the basic suite takes 63 minutes with or without the patch. This
> makes sense, because the effect is likely to be evident mostly in queries
> returning many rows, and those are not very common in the test suite.
>
> I do not have a working Django-trunk project to test on, and this evening and
> tomorrow is a very sacred holiday here (Yom Kippur), so it might take me until
> the end of the week to produce benchmark results. If anyone wants to chime in,
> the pull request is https://github.com/django/django/pull/393. There is, so
> far, anecdotal evidence of very significant improvements, from Ian Kelly
> (original author of this patch) and myself (on a somewhat different
> implementation and older Django versions, but same general idea). In my case,
> page load times were cut by dozens of percents.

Thanks for putting together a pull request.  Please note the patch
will also need to update the documentation to state the new cx_Oracle
version requirement (was 4.3.1, now 5.0.1).

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Oracle backend vs. numbers

[Ian Kelly]

On Tue, Sep 4, 2012 at 3:28 PM, Anssi Kääriäinen
 wrote:
> I did some digging, and the reason for the error is this line in the
> patch:
> self.cursor.outputtypehandler = self._outputtypehandler
>
> What is happening is that self has a reference to self.cursor, which
> has reference to bound method self._outputtypehandler, which has a
> reference to self. Thus, there is a reference loop, and garbage
> collection is delayed. Changing the _outputtypehandler to @classmethod
> seems to solve this problem.

Thanks for tracking that down.  _rowfactory was a module-level
function rather than a method, and I should have left it that way.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Oracle backend vs. numbers

[Ian Kelly]

Resending to the list:

On Sep 4, 2012 11:44 AM, "Ian Kelly"  wrote:
>
> On Mon, Sep 3, 2012 at 6:14 AM, Shai Berger  wrote:
> > On Wednesday 29 August 2012 22:17:22 Ian Kelly wrote:
> >>
> >>
https://github.com/ikelly/django/commit/086f502a1c2acb9db27531f7df78cb9df9a
> >> b83bd
> >>
> > I've run a version based on this -- basically, porting the patch to
Django 1.3
> > which we use (so, no tzinfo), using "float" as the default for
expressions, and
> > putting the whole thing as a derived class from the original adapter
rather
> > than a patch (because that's what we do here, as we also need named
parameters
> > and we had to add them). I found that it leaks something -- the error
says
> > "cursors", but I suspect something more subtle; either way, this is
what I got
> > from a page which makes several hundred queries (don't ask):
>
> Are these ORM queries or custom queries?  If the latter, are you
> explicitly closing the cursors?  That should help prevent the issue,
> although unfortunately the ORM doesn't do this.  The cursors should be
> closed anyway when they're finalized, but maybe we've got a reference
> cycle preventing them from getting cleaned up right away, and too many
> are accumulating between garbage collection passes.
>
> When I've seen this error in the past, the immediate solution has been
> to increase the OPEN_CURSORS parameter in the database.  See:
>
> http://www.orafaq.com/wiki/ORA-01000
>
> Cheers,
> Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django Oracle backend vs. numbers

[Ian Kelly]

On Wed, Aug 29, 2012 at 10:34 AM, Jacob Kaplan-Moss  wrote:
> Hey Shai --
>
> Generally sounds like a good idea, I don't see why this shouldn't go
> in. You might want to wait until someone with more Oracle experience
> chimes in, but I'm having a hard time seeing why a number -> str ->
> number dance would be a good idea.

The reason for it is because cx_Oracle doesn't create Decimal objects
when fetching from the database.  It only creates ints and floats.  As
a result, if you try to use a DecimalField(38), you're not going to be
able to effectively store a 38-digit decimal number, even though
Oracle supports that.  You basically end up being limited to the
precision of a float with some rounding error, since the value gets
converted from NUMBER to float to Decimal.  Using strings as the
intermediate instead of floats allows the full precision to be used.

In theory we could do this only for DecimalFields and leave the other
numeric field types alone, by wrapping each individual DecimalField
expression in the select list in a TO_CHAR() call.  In practice this
work has never been done because of the mess it would make of query
generation, so we instead use a cx_Oracle API feature to transmit all
numbers as strings and then sort it out later.

There's another option that also hasn't been implemented, because it
would mean dropping support for cx_Oracle versions older than 5.0.1.
Perhaps the time has come to do that (I note that the oldest binary
still listed at the cx_Oracle website is 5.1).  This would involve
removing the _rowhandler function and the numbers_as_strings setting
and replacing it with an _outputtypehandler that would only be called
once per column instead of on every single returned value.  I've
written a POC patch for this and so far, performance seems to be much
improved, on par with using cx_Oracle without Django.

https://github.com/ikelly/django/commit/086f502a1c2acb9db27531f7df78cb9df9ab83bd

Shai, if you would be willing to try this patch, I'd like to hear your
results.  Please bear in mind that this is currently only a POC, so it
could be buggy -- I've only run it through the basic tests.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 str.format()

[Ian Kelly]

On Fri, Aug 24, 2012 at 10:30 AM, Daniel Swarbrick
 wrote:
> On 24 August 2012 18:12, Carl Meyer  wrote:
>> Can you link to where in the current docs it specifies that %-formatting
>> is deprecated and/or will be removed? I can't even find, on a cursory
>> search, where it says the new .format() style should be preferred.
>
> It's not easy to find - I've only found the mention of the impending
> deprecation in the Python 3.0 "What's New" docs -
> http://docs.python.org/release/3.0/whatsnew/3.0.html#pep-3101-a-new-approach-to-string-formatting
>
> However, I can't find followup references to that in the Python 3.1
> docs. Maybe the decision was rescinded? Personally I didn't really see
> anything majorly wrong with the %-style formatting. Being a C
> developer myself, it was nice to have some common ground and not have
> to go hunting for format string specifications.
>
> This section http://docs.python.org/library/stdtypes.html#str.format
> states "This method of string formatting is the new standard in Python
> 3, and should be preferred to the % formatting described in String
> Formatting Operations in new code."
>
> I just thought that if the %-style formatting is indeed earmarked for
> removal, maybe we should start using the new format sooner rather than
> later.

This post by Guido is informative on the subject, even though it's
nearly three years old now:

http://mail.python.org/pipermail/python-dev/2009-September/092399.html

The 3.2 docs specifically say, "As the new String Formatting syntax is
more flexible and handles tuples and dictionaries naturally, it is
recommended for new code. However, there are no current plans to
deprecate printf-style formatting."  The 3.3 docs no longer have this
quote, but they also indicate nothing about planned deprecation or
removal.

Until the Python developers announce an actual timeline for removal
(if they ever do), I can't see any reason for Django to be concerned
about which formatting approach to use, apart from the immediate
concerns of style and efficiency.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18557: get_or_create() causes a race condition with MySQL

[Ian Kelly]

On Aug 8, 2012 10:25 AM, "Cal Leeming [Simplicity Media Ltd]" <
cal.leem...@simplicitymedialtd.co.uk> wrote:
>
> I'm not entirely sure that suggesting every query needs to be committed
is the right way forward either, given that you only need to commit once
before get_or_create() is called to prevent the issue.

No, that's not sufficient. The crux of the problem is that the two get
attempts within get_or_create are within the same transaction, and so to
satisfy the requirements of REPEATABLE READ they *must* return the same
results. So it's not enough just to start with a clean transaction; in
order to prevent this you would need to commit or rollback between the two
gets, *within* the get_or_create call. MySQL actually does handle this much
correctly AFAICT. The weirdness is just that it defaults to RR instead of
RC.

Resorting to database-level autocommit would be overkill (and might create
some other issues), but it does solve this particular problem by adding the
necessary commit in the one place it's needed.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #18557: get_or_create() causes a race condition with MySQL

[Ian Kelly]

On Mon, Jul 16, 2012 at 2:18 PM, Cal Leeming [Simplicity Media Ltd]
 wrote:
> Okay - anyone else want to throw their thoughts at this?
>
> Also - messing with the isolation levels on MySQL is really not a great
> idea.. it can cause all sorts of unexpected behavior.

Just a thought -- I don't have MySQL handy to test on, but what if the
savepoint occurred before the first get instead of before the create?
Would rolling back the savepoint then release the row lock and allow
the second get to read the row?

If this works but performance is an issue, then perhaps we could
dynamically order the operations according to the known isolation
level?

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: admin_changelist failing on Oracle

[Ian Kelly]

On Fri, Jun 1, 2012 at 12:29 PM, Vinay Sajip  wrote:
>
> On Jun 1, 6:39 pm, Ian Kelly  wrote:
>>
>> The error is caused by one of the column names in the table.  See the
>> second paragraph at:
>>
>> https://docs.djangoproject.com/en/1.4/ref/databases/#naming-issues
>>
>
> Thanks, that helped. It seems that in this case, the offending name is
> 'date'. I added a db_column='event_date' and all seems well. As this
> is a standard Django regression test, I presume a ticket should be
> raised?

I don't think a whole ticket should be needed for this, just a pull
request.  It's the kind of thing that I would have just committed
without a ticket back when we were using subversion.  I'm a bit out of
touch with process these days, though, so I could be wrong.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: admin_changelist failing on Oracle

[Ian Kelly]

On Fri, Jun 1, 2012 at 11:29 AM, Vinay Sajip  wrote:
> I'm getting an error when syncdb tries to execute the following SQL
> against Oracle 10g XE:
>
> CREATE OR REPLACE TRIGGER "ADMIN_CHANGELIST_EVENT_TR"
> BEFORE INSERT ON "ADMIN_CHANGELIST_EVENT"
> FOR EACH ROW
> WHEN (new."ID" IS NULL)
>    BEGIN
>        SELECT "ADMIN_CHANGELIST_EVENT_SQ".nextval
>        INTO :new."ID" FROM dual;
>    END;
>
> The error is
>
> django.db.utils.DatabaseError: ORA-06552: PL/SQL: Compilation unit
> analysis terminated
> ORA-06553: PLS-320: the declaration of the type of this expression is
> incomplete or malformed
>
> Can anyone suggest what the problem is?

The error is caused by one of the column names in the table.  See the
second paragraph at:

https://docs.djangoproject.com/en/1.4/ref/databases/#naming-issues

It's a frustrating error because the column name is perfectly valid,
and it has no problem creating the table; it just causes an
arbitrary-looking error when you try to create the trigger.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - now available on GitHub

[Ian Kelly]

On Sat, May 26, 2012 at 4:31 AM, Vinay Sajip  wrote:
> Anssi,
>
> Thanks very much for the feedback.
>
>> Both Oracle and MySQL fail to run because of this error (on both 2.7
>> and 3.2):
>>   File "/home/akaariai/Programming/django/tests/django/db/backends/
>> mysql/compiler.py", line 2, in 
>>     from django.utils.itercompat import izip_longest
>> ImportError: cannot import name izip_longest
>
> I will investigate this.

The izip_longest definition was removed from itercompat in revision
b60b45a2a565 (which is fine, since it was only there for Python 2.5
compatibility), but it means that the places that imported it need to
be changed to import it directly from py3 instead.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: New Release of IBM_DB_DJANGO (1.0.4)

[Ian Kelly]

On Tue, May 22, 2012 at 4:07 PM, Anssi Kääriäinen
 wrote:
> I don't know about licensing... I believe the issue is that we don't
> have enough manpower to keep more backends updated in core. Even now
> the Oracle backend is somewhat problematic - there aren't that many
> contributors (or committers for that matter) who have Oracle
> installed, and thus it is somewhat common that commits break that
> backend. Then we fix those errors afterwards when/if we spot the
> errors. Adding another not commonly available backend to the matrix
> would cause more similar problems.

I try to make a point of running the Oracle backend through the test
suite in the lead-up to each minor release and fixing whatever I find.
 Unfortunately, I rarely find the time to do much more than that these
days.  I wonder whether Oracle would even make it into core if the
decision were revisited.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: suggestion: Don't make the tag "url" dirty in Django1.5

[Ian Kelly]

On Tue, Mar 27, 2012 at 1:26 PM, gs412  wrote:
> Less symbol is the tide, the future. for example:
> coffeescript、sass、haml、slim and python, there are all less symbol, this
> feature make developer fell well

This is incorrect.  If Python were all about "fewer symbols", then we
would favor tabs over spaces, we would not require colons at the ends
of block-starting lines, and we would have operators like "!" instead
of "not".

One of the goals of Python is to be *more readable*, not to be
unnecessarily terse.

> passing context variables for tag "url" is not a good idea, it make the work
> from easy to hard
>>
>> {% with url_name="client-detail-view" %}
>>     {% url url_name client.id %}
>> {% endwith %}
>
>
> three lines instead of one lines, just for passing context variables,
> context variables for url is not widespread

You don't need three lines.  There is no reason that the following
shouldn't work, so if it doesn't, then that is something that ought to
be fixed.

{% url "client-detail-view" client.id %}

This syntax has been available since Django 1.3, which was released a
year ago.  If Django 1.5 takes the same amount of time that 1.4 did,
then it is still another year away.  That is two full years to bring
your templates up-to-date, and it is a simple enough change that you
could probably do it with a short script.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: incorrect check email with 'email_re' in 'django.core.validators'

[Ian Kelly]

On Mon, Dec 19, 2011 at 7:20 PM, Wim Lewis  wrote:
>
> On 19 Dec 2011, at 4:50 PM, Nick Fitzsimons wrote:
>> Strictly speaking that regex cannot determine that an email address is 
>> well-formed per the RFC as the grammar defining the form of email addresses 
>> is a Type 2 Chomsky Grammar and regular expressions are limited to Type 3 
>> Chomsky Grammars.
>
> That's true of the whole address line, but is it true of what people usually 
> want to validate in an email-address field (the "addr-spec" production from 
> rfc2822, without any comments)? I think that language is regular, unless 
> there's a recursive rule hidden in the obsolete forms part of the grammar. 
> The comment syntax is type-2, of course, because it requires balanced 
> parentheses.
>
> (And of course, regexes aren't even regular expressions aka type-3 grammars 
> any more--- but not enough more to parse balanced strings, I think.)

I think that it's regular, as there isn't any nesting in the
"addr-spec" rules.  In fact, to write an FSM just for the local part I
think you would only need four states: start-of-string, normal,
quoted-string, and start-of-quoted-pair.  The domain part is similar.
The complexity mostly lies in what characters are accepted in each
state.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: incorrect check email with 'email_re' in 'django.core.validators'

[Ian Kelly]

On Mon, Dec 19, 2011 at 9:36 AM, Maxim Badika  wrote:
> Hello all!
>
> i found that
>
> email_re.search(u'my...@mydomain.com.') # with dot at the end of
> address
>
> returns true. I think that is incorrect.
> Thanks.

That is actually correct, and it is equivalent to "my...@domain.com".
See RFC 3696.

The root of the domain name tree is known as "." and the TLDs are
technically "com.", "edu.", "uk.", etc.  When discussing domains below
the TLDs, the trailing dot is normally omitted ("domain.com"), but
applications are still required to accept it

Cheers,
Ian.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - notes for people wanting to review changes/port apps

[Ian Kelly]

On Dec 17, 2011 10:12 PM, "Yo-Yo Ma"  wrote:
>
> Is this a prank, or is there really going to need to be a u('') for
> every u'' (of which there are thousands, since everything is Unicode
> in Django) in my Django apps, in order to use P3K? Removing the "u" is
> one thing, but adding another text function seems absurd, no?
>
> name = models.CharField(_(u('name'))). # eek :/

The plan is to drop Python 2.5 support on the same timeline as merging
this, so no, you won't need to do that, since 2.6+ support the Python 3
literal syntax with a __future__ import.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.4 alpha on December 22nd

[Ian Kelly]

On Thu, Dec 15, 2011 at 5:00 PM, Kiril Vladimirov  wrote:
> While I'm reading all topics, listed below I'm thinking... is it too early
> to announce *experimental* Python 3.2 support in Django 1.4?

Already been discussed:

http://groups.google.com/group/django-developers/msg/32c3526c4efe278d?hl=en

The problem with merging it and labeling the support as "experimental"
is that the changes are of such a fundamental nature that they could
easily break things in 2.x, not just in Python 3.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all PostgreSQL tests now pass on 2.7.2 and 3.2.2 with the same codebase

[Ian Kelly]

On Sat, Dec 10, 2011 at 1:03 AM, Vinay Sajip  wrote:
> So - with Ian Clelland's post[1] from a couple of days ago, we have
> successful runs on 2.x and 3.x using a single codebase and SQLite,
> PostgreSQL and MySQL backends. I suppose Oracle will be the next one
> to focus on: Ian Kelly was getting 17 failures and 24 errors on 7
> Dec[2], but some changes have been committed since then.

To be clear, I was seeing the same failures using both Oracle and
SQLite.  I suspect they probably had more to do with testing on
Windows than with Oracle.  I'll try to take another look at it this
weekend.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: FileField should be a context manager

[Ian Kelly]

On Fri, Dec 9, 2011 at 11:49 AM, Florian Apolloner
 wrote:
>
>
> On Thursday, December 8, 2011 11:36:53 PM UTC+1, Alex_Gaynor wrote:
>>
>> It isn't alreayd?!?!  All file objects django exposes should be context
>> managers, I thought I added that to a base mixin class a while ago, but
>> apparently I didn't add it to the right thing.
>
>
> Your memory serves you well ;) I wrote the patch and you commited it:
> https://code.djangoproject.com/ticket/14749 We added it only to File though.

FieldFile inherits from File, though, so the methods are present.  You
just have to do:

with obj.file_field as f:
...

instead of calling the open() method (which returns None), as the OP
tried.  The only catch is that the __enter__ method does not open the
file, so if you close the file and then try the above, you'll get an
exception.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.7.2 and 3.2.2 with the same codebase

[Ian Kelly]

On Tue, Dec 6, 2011 at 8:46 PM, Ian Clelland  wrote:
>
>> * compiler.py still had a map(None) call, that I replaced with
>> izip_longest.
>
>>
>> The biggest change was just adding izip_longest to utils.py3 and
>> utils.itercompat.
>
>
> I had the same issue with MySQL -- I used a block like this:
>
> try:
>    from itertools import zip_longest
> except ImportError:
>    # python 2.x
>    from itertools import izip_longest as zip_longest
>
> The code doesn't really seem to mind; iterator or generator, so I didn't
> think it was worth making an exact compatibility function

itertools.izip_longest was added in Python 2.6, though, so a
compatibility function is needed for Python 2.5.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.7.2 and 3.2.2 with the same codebase

[Ian Kelly]

I've just finished going through the oracle tests and sent you a pull
request that fixes a few issues:

* compiler.py still had a map(None) call, that I replaced with izip_longest.
* cx_Oracle doesn't seem to want to accept bools as bind parameters
with Python 3; these had to be converted to ints.
* we were calling iterator.next() in one place instead of next(iterator)
* a couple of tests (one of which is specifically an Oracle test)
needed minor updates

The biggest change was just adding izip_longest to utils.py3 and
utils.itercompat.

With these changes, I'm not seeing any failures with oracle that I'm
not also seeing with sqlite3.  I am seeing a bunch of failures,
though.

FAILED (failures=17, errors=24, skipped=97, expected failures=2,
unexpected successes=1)

Most of the failures are coming from the timezones tests.

Also, one rather obnoxious failure that I encountered was the
file_uploads.FileUploadTests.test_large_upload test, which doesn't
actually fail but just hangs indefinitely.  I had to skip the test in
order to run the full suite.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.7.2 and 3.2.2 with the same codebase

[Ian Kelly]

On Fri, Dec 2, 2011 at 11:34 AM, Joe & Anne Tennies  wrote:
> So, last time I saw, you had only run against sqlite. Do you need help
> testing it against postgres, MySQL, and oracle (perhaps some unofficial
> ones)? What about all the caching backends? Do we need tests Python 3
> equivalent tests for the ones that were skipped due to being Python 2-isms?
> What about some sort of "porting guide" or other needed documentation?
>
> Basically, what's left, besides getting it merged into the official trunk
> that people can help you with?

I'm already planning to go through the Oracle tests this weekend to
get them ready for 1.4, so I'll run this through as well and let you
know how it goes.  Right now I expect failures even in 2.7.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.4 roadmap

[Ian Kelly]

On Tue, Nov 29, 2011 at 5:10 AM, Luke Plant  wrote:
> 2) Broken tests:
>
> http://ci.djangoproject.com/builds
>
> It looks like tests are failing on Oracle and spatialite. I guess any
> failing tests need to be added to the release blockers.

I'll make a note to go through the Oracle failures soon.  Things have
a tendency to break over time in the Oracle tests when I'm not
watching, mainly because people hard-wire their tests to check for
specific primary keys, not realizing that the oracle backend doesn't
do sequence resets in the same way and isn't guaranteed to see the
same primary keys.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Suggested Oracle double-fetch fix (was Re: Can we drop Oracle 9 support?)

[Ian Kelly]

On Thu, Nov 24, 2011 at 3:00 PM, Shai Berger  wrote:
> While we're all at it, I would like to bring to your attention another
> suggested fix for Oracle. The patch attached here tells the backend that if a
> fetch operation retrieved less than the requested array size, then no further
> fetch attempts are necessary.
>
> The fix is motivated by our DBA's finding, that many single-row-selects are
> followed by double fetches. When I checked the code, I realized that with the
> exception of aggregates, this is true for all single-row selects: whether it
> originates from qset.get(...) or from qset[j], the QuerySet code adjusts the
> SQL and then calls fetchmany in a loop until no results are returned. Since
> neither cx_oracle nor the Oracle client library take it upon themselves to
> stop such requests, they go all the way to the database.

There is nothing specific to Oracle about this.  This optimization
should be performed in the query engine, not in the backend.

> The Python DB-API[0] says on fetchmany:
>  """
>    The method should try to fetch as many rows as indicated by the size
>    parameter. If this is not possible due to the specified number of rows not
>    being available, fewer rows may be returned.
> """
> I interpret that as "no rows should be fetched after a call failed to fill the
> buffer". Under a loose-enough transaction isolation level, rows may "come into
> being" in mid-transaction, but I don't think it is reasonable to expect them
> to do so in mid-statement.
>
> That being said, the problem that this patch fixes is a little hard to test
> for; it essentially requires mocking the database library. Also, I'm not sure
> this is the right fix; I don't know if other backends have the same issue. If
> they do, it might be better to make qset[j] just use fetchone instead of
> fetchmany, like the aggregates do; and it might be worthwhile to also change
> the qset.get() logic (in a more subtle way -- we don't want to lose the check
> for multiple records returned).

I think that this is the right approach.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Can we drop Oracle 9 support?

[Ian Kelly]

On Tue, Nov 22, 2011 at 1:54 AM, Anssi Kääriäinen
 wrote:
> I can do a patch for this if needed. Are you (or somebody else)
> working on this?

I had some specific ideas for how to approach it, but I haven't done
any actual work yet.  Feel free to claim it.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Can we drop Oracle 9 support?

[Ian Kelly]

On Wed, Nov 16, 2011 at 4:55 PM, Shai Berger  wrote:
> Hi,
>
> Oracle 10 has been released in 2003 -- the same year as Python 2.3, which is
> no longer supported by Django.
>
> Oracle 9 is essentially end-of-life'd -- Oracle will not provide patches for
> it, even for critical bugs, and even if you pay extra. This is since July
> 2010, according to
> http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf
>
> Support for Oracle 9 is a very minor thing in the backend code itself -- only
> a handful of lines -- but these lines execute a command on the backend to get
> the server version, at every connection, and our data indicates this command
> does affect performance. You might expect that it just looks up a constant,
> but in fact it is a stored procedure that looks in several places to calculate
> not just the software version, but also the interface compliance level.
>
> In the backend, the version info is only used to tell if we're still on 9. If
> we drop 9 support, we don't need that anymore.
>
> For the benefit of those who need to know the version (e.g. 3rd-party code
> which uses specific Oracle 11 features), I propose we change oracle_version
> from an attribute on the connection that is filled at connection time, to a
> property that proxies the cx-oracle connection property. That property is
> memoized (at the connection-object level) anyway, so for code that relies on
> it, the only difference would be in when the server statements are executed,
> while for the rest (most?) of us, the statements will just not be run.
>
> So -- can we drop Oracle 9? Should I open a ticket for it?

-0.  This isn't really like deprecating old Python versions, which is
necessary to keep the codebase unified while gaining access to new
features.  We have a certain subset of SQL that we need to be
supported by a backend, and that subset doesn't grow very quickly.
Django works on Oracle 9 with the one exception of regex query
filters, and that status is unlikely to change with new Oracle
versions.  The main reason to drop Oracle 9 support would be to
support more recent features, and I'm not aware of anything
particularly compelling for use by Django.

I would also want to poll the community before dropping support, to
verify that few users are still using Oracle 9 with Django.  We still
have some Oracle 9 databases where I work, although none running
Django at this point.  I've even (recently) heard of organizations
still stuck on Oracle 8.

The version check that you point out does not really need to be done
on every connection, I think.  If we make a connection and find one
version and then later make a new connection using the same
DatabaseWrapper, we expect the database to still be the same version.
I would propose that instead of dropping support, we make this check
once per DatabaseWrapper per thread, in the same way that the LIKE
operators are already configured.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: queryset caching note in docs

[Ian Kelly]

On Wed, Nov 2, 2011 at 5:05 AM, Anssi Kääriäinen
 wrote:
> For PostgreSQL this would be a nice feature. Any idea what MySQL and Oracle
> do currently?

If I'm following the thread correctly, the oracle backend already does
chunked reads.  The default chunk size is 100 rows, IIRC.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Curious (bug?) with db filter __in

[Ian Kelly]

Have a look at the DatabaseOperations.max_in_list_size method, but it sounds
like it's not going to be a simple constant for MySQL.
On Oct 3, 2011 1:01 PM, "Cal Leeming [Simplicity Media Ltd]" <
cal.leem...@simplicitymedialtd.co.uk> wrote:
> Ahh - max_allowed_packet pops up its ugly head again - it
> wouldn't surprise me if this was the case.
>
> Thank you for testing this against Postgre - I will post the test results
> against MySQL tonight/tomorrow.
>
> Cal
>
> On Mon, Oct 3, 2011 at 7:58 PM, Jacob Kaplan-Moss wrote:
>
>> On Mon, Oct 3, 2011 at 1:31 PM, Cal Leeming [Simplicity Media Ltd]
>>  wrote:
>> > So, came up against a strange thing today.
>> > Database backend is MySQL 5.5 (Percona variant)
>> >
>> > If I attempt to do an __in query with a list containing 50 thousand
>> entries,
>> > the query wouldn't fail, but would instead return no results. If I
split
>> it
>> > up into chunks (say 100) - it would work fine.
>> > For example:
>> > _list = ['user1', 'user2'] # imagine this list has exactly 50 thousand
>> > entries
>> > Members.objects.filter(username__in = _list) # no results
>> > Members.objects.filter(username__in = _list[:100]) # 100 results
>> > I can provide exact further info, but this was just a preliminary email
>> to
>> > see if this was expected behavior - or actually a bug??
>>
>> I'm guessing it's MySQL being "special" again. I tried IN queries with
>> 100k arguments on postgres and it worked fine. The MySQL documentation
>> states that "The number of values in the IN list is only limited by
>> the max_allowed_packet value."
>> (
>>
http://dev.mysql.com/doc/refman/5.5/en/comparison-operators.html#function_in
>> ),
>> so perhaps investigate this max_allowed_packet setting?
>>
>> Jacob
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
"Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: please reopen ticket 15567

[Ian Kelly]

On Tue, Sep 13, 2011 at 11:24 AM, Adam Jenkins  wrote:
> +1 on making the error say more than incorrect username/password. That
> is confusing. In regards to leaking information about the user. The
> error message in general could be changed to something like this, of
> course with better wording:
>
> "Username and password incorrect or access to this page restricted".
>
> The current status is that we are telling the user something this is
> incorrect. I've actually run into this situation before where I had a
> user reset their password a few times before coming to me.

+1 on this suggestion.  This has no security implications and is
clearly an improvement over the existing message.

-1 on the idea of having two separate messages.  It gives away
information, and regardless of whether that information is useful to
an attacker, we should not be trying to predict that.  We can't
envision all possible scenarios, so we should just assume that the
information *is* useful and avoid doing that.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Timezone-aware storage of DateTime

[Ian Kelly]

On Thu, Jun 30, 2011 at 3:10 PM, Aymeric Augustin
 wrote:
> Strictly speaking, converting a naive datetime + a time zone to a "point in 
> time" (eg. UTC timestamp) isn't possible, because of DST: one hour doesn't 
> exist, and another one happens twice. In my opinion, the primary goal of the 
> DateTime field is to store a "point in time" and it should do so 
> unambiguously. Of course, we could store a naive datetime plus a time zone 
> plus a DST flag, but that's cumbersome, I'd even say developer-hostile.

I'm no expert on time zone handling, but shouldn't the two datetimes
already have two different timezones in that case?  For example,
01:30:00 CDT, followed by 01:30:00 CST an hour later, if the user is
in the U.S. Central time zone.  Both of those times should be
convertible to distinct UTC times.  Likewise for the CST to CDT
transition in the spring.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Removal of DictCursor from raw query.. why??

[Ian Kelly]

On Fri, Jun 17, 2011 at 5:54 AM, Cal Leeming [Simplicity Media Ltd]
 wrote:
> Because I feel this is just something that should work (or be available) out
> of the box. There are plenty of other places where Django docs has included
> code snippets to give the user a heads up, and I think this is the perfect
> case for one.
>
> If anyone has any objections to this, please let me know, if not ill put in
> a ticket for consideration.

The thing is, this is a DB API snippet, not a Django snippet
specifically.  If Django were a DB API toolbox, then it might make
sense to include it in some form or other.  But it's not, so in the
interest of keeping things relatively tidy I'm a -0 on this.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Test optimizations (2-5x as fast)

[Ian Kelly]

On Mon, May 16, 2011 at 10:12 PM, David Cramer  wrote:
> Postgres requires resetting the sequences I believe. I just assume
> Oracle/MSSQL are probably similar.

Actually in the Oracle backend, resetting the sequence for an empty
table is currently a no-op for transactional reasons.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: math tag

[Ian Kelly]

On Tue, May 3, 2011 at 12:57 PM, Johannes Dollinger
 wrote:
> * __mul__: Display a value of 0.42 as 42%.

This would be better implemented as a "percentage" filter, IMO.  It
would be a natural candidate for inclusion in django.contrib.humanize.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC #2705 (Support for SELECT FOR UPDATE)

[Ian Kelly]

On Wed, Apr 13, 2011 at 2:08 PM, Dan Fairs  wrote:
> However,
> it's worth mentioning that the Oracle tests seemed to be mostly broken in
> the first place, with dozens of errors.

Thanks for following up on those.  I reviewed the Oracle tests shortly
before the 1.3 release and fixed all the real failures in the backend.
 The remaining failures as of then were due to the backend not
interacting well with the test suite as documented in ticket #15573,
plus a DB cache issue in #15580.

Thanks,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: secret key from file...

[Ian Kelly]

On Tue, Mar 22, 2011 at 5:17 PM, Carl Meyer  wrote:
> Interesting. I would have assumed that the reason is so that code in
> site_settings.py has access to the previously defined values in the main
> settings.py, and can actually modify them (i.e. append to
> MIDDLEWARE_CLASSES or whatnot). With an import this is not possible.

That's a good reason too.  It didn't occur to me because we don't
actually do that, and I'm having a hard time envisioning a scenario
where we might want to.  It's good that it's not outside the realm of
possibility, though.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: secret key from file...

[Ian Kelly]

On Tue, Mar 22, 2011 at 4:49 PM, Matt Robenolt
 wrote:
> Why not just do an import for your custom settings?
>
> try:
>        from site_settings import *
> except ImportError:
>        pass

No particularly compelling reason that I know of, the import machinery
is just unnecessary in this case.  The site_settings.py is viewed as
an extension of the settings.py, so it doesn't need to be loaded as a
module in its own right.  And for the same reason we know exactly
where we expect the file to be, so there's no need to consult
sys.path.

I suppose it just comes down to a matter of taste.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: secret key from file...

[Ian Kelly]

On Tue, Mar 22, 2011 at 6:44 AM, Kristaps Kūlis
 wrote:
>  I personally would greatly appreciate update in docs to show "best
> way" to handle per enviroment settings / sensitive settings, as now
> there is many ways :)
>  Consider when one has his local development enviroment with locmem
> cache and sqlite3 db, and staging server with MySQL / memcache and
> production cluster with MySQL (different credentials) and memcache
> cluster. In settings.py most deployment  specific stuff is left blank
> and on deployment {staging/production}_settings.py is created with
> from settings import * and then overriding per env settings :).
>  Actually it took me far too much googling to find "how" and "what" to
> do, at least I believe that such stuff should be put in docs to make
> django more sysadmin friendly.

I don't know what the best way is, but I will share what we do.  In
addition to the regular settings.py we have site_settings.py (which is
not under version control) in the same directory.  Then at the end of
settings.py we add this simple code:

# Run a separate python file not in version control for database
# settings and other sensitive information.
from os.path import dirname, join
execfile(join(dirname(__file__), 'site_settings.py'))

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: the new SELECT DISTINCT query in 1.3 rc1, and how to turn it off

[Ian Kelly]

2011/3/9 Łukasz Rekucki :
> It's probably messy in general case, but maybe we can try emulating
> this in this particular case. Using an ordinary DISTINCT in a subquery
> seems to solve the issue:
>
>    base_query =
> self.rel.to._default_manager.using(db).complex_filter(self.rel.limit_choices_to).values('id').distinct()
>    queryset = self.rel.to._default_manager.using(db).filter(pk=base_query)

Yes, that looks like it should work.  I expect it's probably less
efficient, but we may just have to live with that.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: the new SELECT DISTINCT query in 1.3 rc1, and how to turn it off

[Ian Kelly]

2011/3/8 Łukasz Rekucki :
> It's also supported by Oracle, AFAIK.

It is not, although it can be emulated using an analytic query.  I
tried adding this to the patch in #6422 some time ago, but I found
that the structure of an analytic query was going to be rather
complicated to shoe-horn that into the query compilation code.  That
was before the SQL compiler was factored out of the query code, and I
haven't gotten around to trying again since.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: 'ascii' codec can't encode characters in position 0-8: ordinal not in range(128)

[Ian Kelly]

2011/2/22 Alexander Bolotnov 

> I created a simple model:
>
> class Menu(models.Model):
> name = models.CharField(null=False,
> blank=False,unique=True,max_length=50)
> url = models.CharField(null=False, blank=False,
> unique=True,max_length=100)
> sortOrder = models.IntegerField(null=False, blank=False, default=0)
> def __unicode__(self):
> return 'name: {0}, url: {1}'.format(self.name, self.url)
>
> and hooked it to admin (all running on windows 7, django dev server)
>
> now when I add a new item via admin and the name is in Russian (Like
> "коллекция картинок") it throws the following exception. How do I fix this?
>

Please direct questions about how to use Django to the django-users mailing
list.  This list is for development of Django itself.

Thanks,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC #2705 (Support for SELECT FOR UPDATE)

[Ian Kelly]

On Tue, Dec 21, 2010 at 10:31 AM, Dan Fairs  wrote:
> I've mainly worked on the tests and docs, not the core content of the patch, 
> but I'd welcome any comments on the implementation as well - I'd like to see 
> this get in (be it in a 1.3 or 1.4 timeframe) so if any changes are needed 
> for that, then that's fine. I'd also appreciate it if someone with access to 
> Oracle could give the patch a try.

I'm on vacation now and don't have easy access to an Oracle setup, but
I'll be happy to try it next week if nobody else has done it first.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3

[Ian Kelly]

On Wed, Dec 15, 2010 at 12:45 AM, sirex  wrote:
> On 15 Gruo, 01:51, Russell Keith-Magee 
> wrote:
>> but we're not planning to actively support Python 3 any time soon.
>
> Why?

Dependencies, for one.  Psycopg2 and MySQLdb do not officially support
Python 3 yet, so the only backends available out of the box would be
oracle and sqlite3.  Also PIL has not been ported, so ImageFields
would be unusable.

More importantly, the core devs don't want to have to support Python
2.4/2.5 and Python 3 simultaneously.  Hence the suggested plan to
begin officially supporting Python 3 once support for 2.5 has been
deprecated.

> Maybe there should be two versions of Django, one that supports Python
> 3 and other that supports Python 2?

No.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django.contrib.admin and null=True

[Ian Kelly]

On Tue, Dec 7, 2010 at 4:26 PM, schinckel  wrote:
> I think this is broken behaviour: NULL values are semantically
> different to empty strings, and in certain cases (I think it was
> IPAddressField, but I will have another look later), it would fail to
> write to the database.
>
> Is there a reason that the admin interface saves what should be a NULL
> value as an empty string? Do I report this as a bug?

The issue with null IPAddressFields in the admin when using PostgreSQL is known:

http://code.djangoproject.com/ticket/5622

In the more general case, the Django philosophy on NULL vs. the empty
string is as explained in the link that nasp posted.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Is their much benefit In using a second hidden salt

[Ian Kelly]

On Tue, Dec 7, 2010 at 2:27 PM, andy  wrote:
> However I'm a bit curious about the significance of adding a second
> salt to the password before it is hashed and then using the regular
> per-user salt. Currently my opinion is that their is added benefit
> since it make dictionary attacks more challenging and possibly almost
> impossibly if the attacker does not know the hidden salt. Django has a
> secretKey in the setting file I wondering why this could not have been
> used as second salt in the authentication system.

The problem with this is that if you ever have to change your secret
key (e.g., your settings.py file is compromised), then all passwords
will be invalidated.  And not in a friendly way, either.  With
per-user salts, if you need to invalidate a user's password, you can
allow them to log in with the old password and then require them to
change their password.  With a secret key in the salt, if you change
the secret key, then the old passwords will no longer work at all.

Some users may not mind this, but it would be undesirable for core.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #14770: Set cookies from request: Design decision needed

[Ian Kelly]

On Wed, Nov 24, 2010 at 1:11 PM, Jacob Kaplan-Moss  wrote:
>> In my
>> case I was creating a small framework for registering functions that
>> would check for and set messages via the messages framework. If I
>> called the view and passed the request and response to these
>> functions, the messages they'd set wouldn't show until the next
>> response because the page would already have been rendered. If I could
>> stick to only passing in a request then I could put off getting the
>> response object until later, my functions could set cookies, and the
>> messages would show up at the right time.
>
> I'm really not following this "next response because the page has
> already been rendered" thing -- if you create the reponse object, pass
> it into your utility functions, then return it that creates the
> cookies just fine.

I gather that the OP is trying to do something like this:


def some_view(request):
f(request)
return some_response(request)

def f(request, response):
response.COOKIES.set(...)
messages.add_message(request, messages.INFO, "Cookie set")


The problem then is that function f needs the response object in order
to set the cookie, but if the response object is passed into f, then
it is too late to add the message.  This makes it difficult for the
same function to both set a cookie and add a message, unless it is
also responsible for creating the response.

An alternate solution might be to use delayed execution to set the cookie:

def some_view(request):
closure = f(request)
response = some_response(request)
closure(response)
return response

def f(request):
messages.add_message(request, messages.INFO, "Cookie set")
def closure(response):
response.COOKIES.set(...)
   return closure

Although this could cause confusion if the message gets added, and
then the view fails in some way before the cookie actually gets set.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Settings for a pure Oracle multi-db Django test suite setup

[Ian Kelly]

On Mon, Nov 8, 2010 at 10:02 PM, Ramiro Morales  wrote:

> Has anybody had success with this setup i.e. one where the 'default and
> 'other'
> alias pointing to such an Oracle product/install?. Is it possible at all?
>
> If so, could you please share the relevant Django settings file (not
> really, only
> the DATABASES setting)?
>

Yes, here are the settings I'm using:

DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'xe',
'USER': 'ikelly',
'PASSWORD': 'ikelly',
'OPTIONS': {'threaded': True},
'TEST_USER': 'django_test_default',
'TEST_TBLSPACE': 'django_test_default',
'TEST_TBLSPACE_TMP': 'django_test_default_temp',
},

'other': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'xe',
'USER': 'ikelly',
'PASSWORD': 'ikelly',
'OPTIONS': {'threaded': True},
'TEST_USER': 'django_test_other',
'TEST_TBLSPACE': 'django_test_other',
'TEST_TBLSPACE_TMP': 'django_test_other_temp',
},
}

The key is that NAME should be the same ('xe') for both entries, since
you're really establishing two connections to the same database.  The
TEST_USER, TEST_TBLSPACE, and TEST_TBLSPACE_TMP entries must be different,
however.

Cheers,
Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Invalid SQL generated by objects.all()[:1]?

[Ian Kelly]

On Thu, Jul 15, 2010 at 12:04 PM, Alex Gaynor  wrote:
> On Thu, Jul 15, 2010 at 1:00 PM, Mark Bucciarelli  wrote:
>> Hi,
>>
>> I've just been informed by a MonetDB developer
>> that LIMIT and ORDER BY clauses are not
>> allowed by the SQL spec in sub-selects.
>>
>> I filied a bug with them because the Django
>> code:
>>
>>   Event.objects.all()[:1]
>>
>> was failing because of invalid SQL.
>>
>> I don't have access to the SQL spec so I
>> can't quote you verse and chapter.
>>
>> Has this come up before?
>>
>> How can I workaround this in the adapter?
>>
>> Thanks,
>>
>> m
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To post to this group, send email to django-develop...@googlegroups.com.
>> To unsubscribe from this group, send email to 
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at 
>> http://groups.google.com/group/django-developers?hl=en.
>>
>>
>
> It may be that they aren't allowed by the spec, but the only database
> (that Django includes an adapter for) that doesn't support them
> (AFAIK) is MySQL.  Further, Event.objects.all()[:1] doesn't generate a
> subquery at all.

Er, I'm pretty sure that MySQL does support both ORDER BY and
LIMIT/OFFSET.  At least, it is documented as supporting them.

The supported database that does have a problem with LIMIT/OFFSET is
Oracle.  The backend contains a workaround, which is implemented by a
custom SQLCompiler in django.db.backends.oracle.compiler, and it
involves wrapping the original query in a subquery.  The MonetDB
backend may be doing something similar, which would explain why that
query is generating a subquery.

All of the internally supported databases support ORDER BY, so I can't
offer any hints there.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: TimeField for duration

[Ian Kelly]

On Wed, Jan 20, 2010 at 5:33 PM, Alex Gaynor  wrote:
> Ian,
>
> FWIW there's a ticket with a patch (of unkown quality :P) for support
> for using timedeltas with F() expressions.
>
> Alex

I know, I wrote the Oracle backend code for it. :-)

I'm not talking here about passing in literal timedeltas to be added
to TimeFields; I'm talking about adding a TimeField to an
IntervalField implemented as a BigIntField.  From a user's standpoint,
if you can do one, then the other should work as well, but that's not
the case.

Cheers,
Ian
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: TimeField for duration

[Ian Kelly]

2010/1/20 Łukasz Rekucki :
> 2010/1/21 Jerome Leclanche :
>> Hi Lukasz, thanks for the wrap up
>>
>> Main issue is precision. I personally very, very often work with
>> milliseconds, and I can imagine many use cases where I'd need to work
>> with microseconds. How many of these backends do not at least support
>> millisecond precision?
>
> From my understanding of the docs PostgreSQL can do microseconds
> (6digits), Oracle - nanoseconds (9digits), SQLite3 seems to accept
> miliseconds in string representation "HH:MM:SS.SSS".
>
> And the worst case is MySQL:
>
> """A trailing .uu microseconds part of TIME values is allowed
> under the same conditions as for other temporal values, as described
> in Section 10.3.1, “The DATETIME, DATE, and TIMESTAMP Types”. This
> includes the property that any microseconds part is discarded from
> values stored into TIME columns."""
>
> After seeing all this mess, i'm +1 on using BigIntField. Good work.

In other words, all the same limitations that we already handle for
TimeFields.  This looks to me like a green light to just use the
native datatypes.

I see another big downside to the BigIntField approach: you can't add
TimeFields and IntervalFields at the database level (e.g. with F
expressions).  I'm not sure how PostgreSQL or MySQL would handle this
if you tried it, but I know what Oracle would do: it would accept the
SQL as valid, it would interpret the IntervalField value as a decimal
number of /days/, and it would return a completely incorrect result.

Cheers,
Ian
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Ticket #5025 - truncate filter, why hasn't it been accepted?

[Ian Kelly]

On Wed, Dec 30, 2009 at 11:50 AM, David Zhou  wrote:
> Using CSS to truncate email addresses defeats the purpose of
> truncating email addresses.  Not exactly "better", is it?

That depends on whether your purpose is to make it fit in the space
allotted, or to obfuscate the address.  I had understood it to be the
former.  For the latter, using truncate isn't a good solution either.
What if the address happens to be shorter than the truncated length?

Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Ticket #5025 - truncate filter, why hasn't it been accepted?

[Ian Kelly]

On Wed, Dec 30, 2009 at 11:21 AM, Jerome Leclanche  wrote:
>
> On Wed, Dec 30, 2009 at 8:02 PM, Ian Kelly  wrote:
>>
>> What I haven't seen yet for this filter is a clear use case.  If
>> you're just trying to get something working quickly, then slice is
>> fine.  When you're ready to go back and do it right, then the optimal
>> solution is to use CSS [1].  If you only want to truncate at word
>> boundaries, then you'll just use truncate_words.  What requirement
>> does a truncate filter satisfy that these other solutions don't?
>>
>> [1] http://mattsnider.com/css/css-string-truncation-with-ellipsis/
>>
>> Ian
>
> Then perhaps you should read the past few mails, I've given 5 different use
> cases.
> J. Leclanche / Adys

All of which could be handled just as well or better using CSS, unless
there's something I'm missing, which is the reason I asked.

Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Ticket #5025 - truncate filter, why hasn't it been accepted?

[Ian Kelly]

On Wed, Dec 30, 2009 at 10:25 AM, Nic  Pottier  wrote:
> On Dec 30, 8:55 am, Alex Gaynor  wrote:
>> I'd highly recommend watchinghttp://www.youtube.com/watch?v=tscMnoS4YU8in it 
>> this *exact* question
>> comes up and Russ, Malcolm, and a few other people discuss the pros
>> and cons of adding new template tags/filters.
>>
>> Alex
>
> Thanks, that's a useful link.  The relevant portion starts at ~15:00
> for those that are interested.
>
> But really, the sum total of that discussion with regards to truncate
> seemed to be:
>  1) truncate doesn't exist because it wasn't useful in Journalism
>  2) you can add it yourself
>  3) we cover 80/20
>
> What it explicitly doesn't say is that there is a huge cost to having
> a new filter in core.
>
> Don't get me wrong, I absolutely agree on the 80/20 principle, and on
> that a clean and simple interface is hugely valuable. (The PHP mess is
> a clear counterpoint here)
>
> But I disagree that truncate is somehow an esoteric filter.  RoR has
> it, Smarty has it, as do countless others, it just isn't that
> unusual.

What I haven't seen yet for this filter is a clear use case.  If
you're just trying to get something working quickly, then slice is
fine.  When you're ready to go back and do it right, then the optimal
solution is to use CSS [1].  If you only want to truncate at word
boundaries, then you'll just use truncate_words.  What requirement
does a truncate filter satisfy that these other solutions don't?

[1] http://mattsnider.com/css/css-string-truncation-with-ellipsis/

Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Bug #11017: Oracle LIKEC query doesn't use index

[Ian Kelly]

On Tue, Dec 22, 2009 at 12:58 AM, Jani Tiainen  wrote:
> Escaping bug exists in Oracle 9.2.0.5 and earlier. It was fixed in
> patchset 9.2.0.6 and any later version.
>
> What comes to Oracle official support options for 9.2:
> Premier Support Ends: 31-Jul-2007
> Extended Support Ends: 30-Jul-2010
>
> If any problem is encountered you're first required to upgrade to latest
> patchset (9.2.0.8) before requesting further assistance from Oracle. And
> Oracle has always advised to apply patchsets to db.
>
> I have only 9.2.0.1, but bug should appear there and I've patchsets for
> 9.2.0.6 and up so I can see does it really happen after upgrade. I can
> stage and run tests to see was it that or something else.
>
> And known workaround (that works "everywhere") is use one I pasted few
> ticket comment before.
>
> It's more like design and support decision here - is there need to
> support (obsolete) Oracle version(s)?

Hi Jani,

The only thing I need before committing the workaround is for somebody
to verify that the bug in #5985 is reproducible in version 9.2.0.5 or
earlier, and that the workaround indeed fixes it.  Since I don't have
an instance with the appropriate patchset available to test it myself,
I'd appreciate it if you would do that testing.

I for one don't see a compelling case for dropping support for Oracle
9 yet.  If the workaround is successful, then the only Django feature
currently not supported for Oracle 9 is the regex filter, and I'm not
aware of anything on the horizon that will change that.  Oracle shops
also tend to be notoriously slow at updating.  My employer is not as
bad as some, but even we still have some Django installations running
on Oracle 9.

Even if we do decide to drop support for Oracle 9 (or just the early
patchsets thereof) soon, it will take a full release cycle so that we
can issue the appropriate deprecation warnings.  We'll still want to
be using the workaround at least for the short term.

Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Oracle backend TextField unique=True error

[Ian Kelly]

On Mon, Dec 21, 2009 at 4:44 AM, Jani Tiainen  wrote:
> On Mon, 2009-12-21 at 02:42 -0700, Ian Kelly wrote:
>> Not currently, no.  And if I were to put in some work on improving on
>> the Oracle backend's support for filtering on TextFields, I would
>> concentrate first on fixing the query so that it correctly compares
>> the entire TextField and not just the first 4000 characters.
>> Unfortunately, I don't think any function-based index will help with
>> that.
>
> But there exists dbms_lob package to help with that, specially this
> particular search case "dbms_lob.instr".
>
> Of course it still has some issues (PL/SQL limitations) that may apply.
>
> Oracle is very evil what comes to long text fields, there is just not
> good and support for them.

That's the idea.  If we could get it using dbms_lob.instr instead of
dbms_lob.substr, then it would at least be free of the 4000-character
limitation.  But we can't index off of dbms_lob.instr, because it
takes the search string as a parameter.

Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Oracle backend TextField unique=True error

[Ian Kelly]

On Sun, Dec 20, 2009 at 11:25 PM, Mario Briggs  wrote:
>> This is known.  The Oracle notes [1] mention that TextFields cannot be
>> indexed.  Since Oracle requires indexes for unique columns, this also
>> means they cannot be unique, although we should probably make that
>> explicit.  It may also be worthwhile to check for this when the models
>> are validated.
>>
>> Ian
>>
>> [1] http://docs.djangoproject.com/en/1.1/ref/databases/#textfield-limitations
>
> Ian,
>
> From the other post on admin views using Text, it surfaced that Oracle
> (the DB) does support function based indexes on LOB upto ~4K. So the
> backend does not support that i guess ?

Not currently, no.  And if I were to put in some work on improving on
the Oracle backend's support for filtering on TextFields, I would
concentrate first on fixing the query so that it correctly compares
the entire TextField and not just the first 4000 characters.
Unfortunately, I don't think any function-based index will help with
that.

Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Oracle 11g unicode test failure?

[Ian Kelly]

We just changed our test database to Oracle 11gR2 and discovered that
the unicode test case in the basic test is now failing.  This may just
be something wrong with our installation, because we're seeing the
same problem performing the same test through sqlplus, whereas it
works agains Oracle 10g XE.  It happens on two different 11g
databases, however, and I wanted to check whether anybody else with an
11g database is seeing the same thing.

The failure looks like this:

File "/home/ikelly/projects/django.trunk/tests/modeltests/basic/models.py",
line ?, in modeltests.basic.models.__test__.API_TESTS
Failed example:
Article.objects.get(pk=a.id).headline
Expected:
u'\u6797\u539f \u3081\u3050\u307f'
Got:
u'\xbf\xbf \xbf\xbf\xbf'

Thanks,
Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Oracle backend TextField unique=True error

[Ian Kelly]

On Fri, Dec 18, 2009 at 9:00 AM, Peter Herndon  wrote:
> Hi all,
>
> I've run into a situation where syncdb produces an error on one of my models 
> against Oracle, but not against Postgres.  Using Postgres, one can mark a 
> TextField as unique=True, and syncdb works just fine.  Using Oracle, you get 
> an error:
>
>> cx_Oracle.DatabaseError: ORA-02329: column of datatype LOB cannot be
>> unique or a
>> primary key
>
> The error is occurring for someone using some code of mine 
> (django-ldap-groups), rather than something I'm seeing directly.  I don't 
> currently have access to an Oracle instance to test against.  FWIW, I 
> remember running into this sort of limitation back when I used Oracle a few 
> years ago, any LOB field was basically opaque -- no indexing, no uniqueness, 
> etc.
>
> Is this limitation known?  Is this an error on my part, that I shouldn't be 
> trying to mark a TextField as unique?  Or should this be something that 
> should be specially handled by the Oracle backend somewhere?  If the latter, 
> I'm happy to file a bug report.

This is known.  The Oracle notes [1] mention that TextFields cannot be
indexed.  Since Oracle requires indexes for unique columns, this also
means they cannot be unique, although we should probably make that
explicit.  It may also be worthwhile to check for this when the models
are validated.

Ian

[1] http://docs.djangoproject.com/en/1.1/ref/databases/#textfield-limitations

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Last call: #11863 (Model.objects.raw)

[Ian Kelly]

On Tue, Dec 15, 2009 at 4:31 PM, Jeremy Dunck  wrote:
> RawQuery.validate_sql excludes anything but selects, but Oracle uses
> comments to do query hinting.  Can an Oracle person confirm that those
> hints can't start the query SQL?  (Not worth holding up landing, of
> course.)

Hints always immediately follow the SELECT keyword.

http://download.oracle.com/docs/cd/B28359_01/server.111/b28286/sql_elements006.htm#i31713

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Oracle/GIS Testers Needed

[Ian Kelly]

On Thu, Nov 26, 2009 at 8:38 AM, Russell Keith-Magee
 wrote:
> Yes - this does help. The confirms my suspicion - the problem is that
> the fixture loaded in FixtureTestCase.test_fixture_loading isn't being
> cleared before running the tests in QueryTestCase.
>
> The annoying thing is that I don't see this problem on any other
> database, so there is something specific about transaction rollback on
> Oracle. I can't see anything in that code that is should be
> Oracle-specific behavior.
>
> The magic code is in _fixture_teardown(), starting at line 487 of
> django/tests/testcases.py. That method *should* be resetting the
> database at the end of multiple_database.FixtureTestCase, but
> evidently it isn't. Either the transaction rollback code isn't getting
> invoked, or it is getting invoked but not behaving as expected.
>
> Any debugging assistance you can provide here would be most welcome.
>
> Yours
> Russ Magee %-)

It sounds like you're running into another case of #11101.  The basic
problem is that loading fixtures involves resetting sequences, which
requires running DDL, which causes an implicit commit.

Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Problem with history view in admin page

[Ian Kelly]

On Mon, Nov 23, 2009 at 11:00 PM, Russell Keith-Magee
 wrote:
> On Tue, Nov 24, 2009 at 1:07 PM, Mario Briggs  wrote:
>> I agree that INTEGER is not the right choice, but then so too is CLOB.
>> How long is this string-serialized representation going to be? greater
>> than 4000 characters ? Varchar(X)  where X is > 4000 or something is
>> then the right choice. This is validated by what Karen says is the
>> Oracle fix.
>
> Well, Django doesn't make the decision to use CLOB - that's in the
> hands of your backend. In the same circumstances, SQLite and Postgres
> use 'text'. MySQL uses 'longtext'. Oracle uses 'NCLOB'

The decision is based on the field type, not on the individual field.
If it were possible within the Oracle backend to override this
particular field to VARCHAR2, we would happily do it.  But TextFields
in general need to be LOBs.

> In theory, the contents of the object_id field could be anything -
> including a string of arbitrary length (i.e., a TextField). However,
> in practice, I would be surprised to see 4000+ characters for
> object_id - most primary keys are going to be integers, and the ones
> that aren't are likely to be short strings or string-like datatypes.

Due to the hackish way the Oracle backend implements lookups on LOBs,
it will still fail if the object_id is actually longer than 4000
characters.  It might even raise an error (I haven't tried it).  So at
least in Oracle we gain nothing by making this a TextField rather than
a CharField(max_length=4000).  Plus, if it were a CharField, we would
be able to index it.

Correct me if I'm wrong, but I think the only way to get an object_id
so large that it won't fit inside a backend-dependent CharField is if
the primary key of the model is itself a TextField.  If you ask me,
anybody foolish enough to use a TextField as a primary key deserves
what they get.  Oracle won't even allow it.

Regards,
Ian

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Obtaining group members?

[Ian Kelly]

On Wed, Aug 26, 2009 at 10:36 AM, Francis Reyes wrote:
>
> Hi all
>
> I'm doing some custom permissions in modelAdmin and I need to obtain a
> list of members from a the group . How can this be done?
>
> Thanks
>
> FR

Please ask questions about using Django on the django-users mailing
list.  This list is for discussion about the development of Django
itself.

Thanks,
Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Revisit Ticket 6064?

[Ian Kelly]

On Wed, Aug 19, 2009 at 8:58 AM, Ian Kelly wrote:
> Besides which, the purpose of using the class as the sender is to be
> able to subscribe to only certain types of connections [1].  Using the
> connection instead would defeat that.  That doesn't mean we can't pass
> the connection as an additional keyword argument, and with multi-db
> around the corner there may be a use case for doing that.  If so, it
> should certainly be a new ticket.

Sorry, forgot to attach my footnote:

[1] 
http://docs.djangoproject.com/en/dev/topics/signals/#connecting-to-signals-sent-by-specific-senders

Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Revisit Ticket 6064?

[Ian Kelly]

On Wed, Aug 19, 2009 at 5:44 AM, gthank wrote:
>
> The accepted solution for Ticket 6064 does not seem to address the
> original problem—initializing a *specific* connection—because it uses
> the the class instead of the instance as the sender of the signal. I
> attempted to reopen the ticket but it was closed and I was told to
> take it up on the users mailing list. That didn't seem like the right
> place, but I did anyway and got no response. Perhaps this is the right
> place. I'm attaching a change that uses the ``connection``, instead of
> the ``__class__``, as the sender of the signal. If there is something
> wrong with the changes, I'd be more than happy to revise it, but I'd
> like to have this functionality in Django proper, since it makes my
> particular use case much easier.
>
> My Use Case
> ==
>
> A multitenant application where each tenant has a completely isolated
> set of data. To avoid having to add some sort of tenant foreign key on
> every table and filter on it on every request, I'm using a different
> Postgres schema for each tenant. Therefore, I need to initialize the
> connections I create with a different search_path depending on which
> user made the request.

Currentlty there's no need to pass the connection into the signal
since, as the person who re-closed the ticket pointed out in a code
snippet, it's possible to obtain the connection using an import.

Besides which, the purpose of using the class as the sender is to be
able to subscribe to only certain types of connections [1].  Using the
connection instead would defeat that.  That doesn't mean we can't pass
the connection as an additional keyword argument, and with multi-db
around the corner there may be a use case for doing that.  If so, it
should certainly be a new ticket.

Regards,
Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: #6148. Should database schemas be created as well?

[Ian Kelly]

On Mon, Aug 17, 2009 at 8:19 AM, kmpm wrote:
>
> I'm working on issue http://code.djangoproject.com/ticket/6148 and
> have some questions.
>
> If a db_schema is defined, should it be created automatically or
> should the user be required to set this up in the same fashion as he
> is required to set up the database.
>
> For postgres it might not be any trouble to create the schema within
> the pre existing database but the MySQL implementation of schemas is
> to create a additional database for the schema (CREATE SCHEMA is a
> alias of CREATE DATABASE). This seems a bit to intrusive for my taste
> and that's why I'm asking.

Also in Oracle, creating a schema means creating a user, which has
security implications.  My opinion is that we should not be actively
meddling with schemas, just as we do not with databases, users, or
tablespaces.

Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: ORACLE QuerySet offset

[Ian Kelly]

On Tue, Jul 28, 2009 at 6:20 AM, slafs wrote:
>
> Hi there!
>
> I think I found a little bug in
> http://code.djangoproject.com/browser/django/trunk/django/db/backends/oracle/query.py#L132
> Offsetting the QuerySet (like qs[2:4]) on Oracle backend mixes the
> ordering in the final result set.
> Author claims that "This is the canonical way to emulate LIMIT and
> OFFSET on Oracle." but looking a bit through the web You can find that
> more popular (appropriate?) way is the RANK() OVER function
> (http://www.techonthenet.com/oracle/functions/rank.php)
> so instead SELECT ROWNUM AS "_RN" in line 132
> I would write SELECT RANK() OVER(ORDER BY ROWNUM ASC) AS "_RN", but it
> needs some testing though.
> Do You agree with me?
> Should I open a ticket or something?
>
> Best regards
> Sławek

Hi Slawek,

We used to do it this way [1] until a contributor pointed out that
ROWNUM scales better [2].  I think that comment actually dates back to
the original implementation, so I would take the word "canonical" with
a grain of salt.

Please go ahead and open a ticket for this.  But I'd like to know: are
you encountering problems with ordered queries, or does the mixed
ordering bug only occur with unordered queries?  Also, when this
happens, is the pagination consistent, i.e. if you combine all the
pages together, does each row appear exactly once?  As long as it only
affects unordered queries and the pagination is consistent, I think
this can probably be fixed in documentation.

Thanks,
Ian

[1] But with ROW_NUMBER() instead of RANK() -- the problem with RANK()
is that if two rows compare equally in the ordering, it returns the
same value for both.  As a result, the query might return more rows
than requested.

[2] http://code.djangoproject.com/ticket/9136

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Oracle NLS sorting

[Ian Kelly]

On Fri, Jun 5, 2009 at 2:05 PM, Antoni Aloy wrote:
> I think in your case is quite the same, take a look at
> http://www.oracle.com/technology/tech/globalization/htdocs/nls_lang%20faq.htm
> and the set the enviornment variable NLS_LANG on your application. In
> the setting perhaps.

This won't work, because the Oracle backend will override the NLS_LANG
setting with ".UTF8" when it loads.  There may be a use case for
changing it to only override the character set part of the variable.

Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Oracle NLS sorting

[Ian Kelly]

On Fri, Jun 5, 2009 at 1:29 PM, slafs wrote:

> another thing is that I don't understand why NLS_TERRITORY has to be
> set to 'AMERICA'.

Because the region determines how the days of the week are numbered.
I have no idea what mapping POLAND uses, but if it's different from
AMERICA, then expect to get incorrect results from the 'weekday'
lookup.

> I know it's not quite elegant to fix a source code of Django (but I
> don't how can I subclass a db backend and then use it in my Django
> project - maybe someone can guide me to some docs?).
> So my question(?) would be:
> Shouldn't this be fixed?

We could add keys to the DATABASE_OPTIONS setting that would cause
this to be set.  I would be concerned about allowing NLS_TERRITORY to
be changed in this manner unless we also implement a different
solution for the weekday inconsistency.

Regards,
Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Oracle savepoints

[Ian Kelly]

On Sun, May 17, 2009 at 2:50 AM, Richard Davies
 wrote:
>
> Matt Boersma wrote:
>> I think Malcolm implemented this in the Oracle backend rather than Ian
>> K. or myself.
>
> It was Ian Kelly, I think in [10022].
>
>> So I'd say the answer is a).
>
> If that's right, then we have an inefficiency on Oracle at present -
> the uses_savepoints flag is used both to signal that savepoints are
> implemented (see BaseDatabaseWrapper._savepoint*() in django/db/
> backends/__init__.py), and to signal that they are needed to wrap code
> which may throw IntegrityErrors (as per reason b; see
> QuerySet.get_or_create() in django/db/models/query.py).
>
> We should probably be splitting these two meanings, so Postgresql
> would have:
>
> can_savepoint = True
> needs_savepoint_after_exception = True
>
> Whereas Oracle would have:
>
> can_savepoint = True
> needs_savepoint_after_exception = False
>
> I'll open a ticket to remember this once Ian, Malcolm or another also
> confirms.

Yes, it was reason a).  Transactions aren't automatically invalidated
after an IntegrityError, as far as I was able to determine.

Regards,
Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Quoting in extra(select=...) expression

[Ian Kelly]

On Mon, May 11, 2009 at 12:33 PM, lemming110  wrote:
>
> I am trying to use the select keyword in extra.  But I cannot properly
> quote the the expression for postrgres.  I am using the
> django.contrib.comments.  I started with this snippet
> http://www.djangosnippets.org/snippets/1101/ which returns the number
> of comments for an object.  However, the code does not work.  The
> problem that the field django_comment.object_pk is text and my item pk
> is integer.
>
> In postgres, this works:
>
> select count(*)
> from django_comments
> where django_comments.content_type_id=12 and
> django_comments.object_pk='332';
>
> But using extra(select={'comment_count': sql,}) where
>
> sql = '''select count(*)
> from django_comments
> where django_comments.content_type_id=12 and django_comments.object_pk=
> %s'''
>
> gives me an error every way that I have tried to add quotes.  (This
> includes django.db.connection.ops.quote_name.)
>
> Any ideas on how to embedded quotes in postgres?

Please ask questions about how to use Django on the django-users
mailing list.  This list is for discussion about the development of
Django itself.

Thanks,
Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: (Discussion) Ticket 9986: CACHE_BACKEND argument name doesn't match code

[Ian Kelly]

On Mon, Mar 30, 2009 at 2:22 PM, Marty Alchin  wrote:
> Personally, I'd rather see an argument name that tries to tell users
> what to supply, rather than solely trying to be true to the algorithm.
> Obviously, the goals should coincide, but focusing on the supplied
> value brings another option to my mind: cull_passes (or something
> similar, like cull_sweeps). Essentially document it as being the
> number of times the cull operation would need to be performed to
> completely empty the cache.
>
> That way, we could describe what the cull operation actually does,
> rather than how it works. If you want it to cull a lot, set it to a
> low number so that fewer passes are required to clear the whole thing.
> Setting it to a higher number would require more passes to do the same
> job. Does that make sense to anyone else?

The only thing that bothers me about this approach is that running the
cull operation multiple times in a row wouldn't actually do that.  If
you set the parameter to 2, and you run it twice, the first sweep
would cull half the entries, and the second sweep would cull half the
remaining entries, leaving 1/4 of the original entries in the cache.
But since culling multiple times in a row isn't a real use case, it
probably doesn't matter.

Ian

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---