Re: Database pooling vs. persistent connections
It is most definitely not an "error" to have less connections available than workers, considering workers may serve different types of requests, and will now persist the database connection even after that request has finished. It may not be "the most correct", but a concurrent connection is very different than a single operation. On Thursday, February 28, 2013 at 1:40 PM, Michael wrote: > On Thu, Feb 28, 2013 at 4:10 PM, Christophe Pettus (mailto:x...@thebuild.com)> wrote: > > > > On Feb 28, 2013, at 11:09 AM, David Cramer wrote: > > > > > Immediately for anyone who has configured more workers than they have > > > Postgres connections (which I can only imagine is common among people who > > > havent setup infrastructure like pgbouncer) things will start blowing up. > > > > If they have this configuration, it's an error. The fact that the error is > > now surfacing doesn't make it a correct configuration. > > Why is this an error? It sounds more like a barrier to entry to me. Most > sites won't ever hit the maximum number of database connections, but they > over provision in their app (or they add more apps to the database as they > see their database load isn't so high). Seems to me if we enable this by > default, we are requiring a certain set of hardware that is way more > demanding than the actual use case for the app, just for a small speed > increase. > > -- > You received this message because you are subscribed to a topic in the Google > Groups "Django developers" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/django-developers/NwY9CHM4xpU/unsubscribe?hl=en. > To unsubscribe from this group and all its topics, send an email to > django-developers+unsubscr...@googlegroups.com > (mailto:django-developers+unsubscr...@googlegroups.com). > To post to this group, send email to django-developers@googlegroups.com > (mailto:django-developers@googlegroups.com). > Visit this group at http://groups.google.com/group/django-developers?hl=en. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Database pooling vs. persistent connections
Can we please change this so it defaults to off, and just document how to turn it on and in what situations you should turn it on? In my opinion this default-on feature caters to a very specific audience, and will cause a lot of unexpected behavior with other users. Here is the tl;dr of an argument for turning it on: Connections are expensive, we should persist them. Here is the tl;dr for turning it off: Connections are expensive, so we dont have a lot of them. Immediately for anyone who has configured more workers than they have Postgres connections (which I can only imagine is common among people who havent setup infrastructure like pgbouncer) things will start blowing up. Why should this be the default behavior? Unlike MySQL, which already has cheap connections and doesn't suffer this problem, connections in Postgres add quite a large cost, both on creation, on allowing them. **Everyone** who cares about their performance on Postgres should already be using pgbouncer (or some alternative) to offset **both** of these, not just a single problem which is what Django is addressing here. I will happily defend my opinion in person at PyCon in two weeks if it still remains a default, and anyone is willing to listen, and if you reach out to the other large (and I dont mean scale) Django users I imagine you will find a lot of mixed feelings about this default. On Sunday, February 17, 2013 3:24:52 AM UTC-8, Aymeric Augustin wrote: > > **tl;dr** I believe that persistent database connections are a good idea. > Please prove me wrong :) > > > > Since I didn't know why the idea of adding a connection pooler to Django > was > rejected, I did some research before replying to the cx_Oracle > SessionPooling > thread. > > The best explanation I've found is from Russell: > > > To clarify -- we've historically been opposed to adding connection > > pooling to Django is for the same reason that we don't include a web > > server in Django -- the capability already exists in third party > > tools, and they're in a position to do a much better job at it than us > > because it's their sole focus. Django doesn't have to be the whole > > stack. > > All the discussions boil down to this argument, and the only ticket on the > topic is short on details: https://code.djangoproject.com/ticket/11798 > > > > The connection pools for Django I've looked at replace "open a connection" > by > "take a connection from the pool" and "close a connection" by "return the > connection to the pool". This isn't "real" connection pooling: each worker > holds a connection for the entire duration of each request, regardless of > whether it has an open transaction or not. > > This requires as many connection as workers, and thus is essentially > equivalent to persistent database connections, except connections can be > rotated among workers. > > Persistent connections would eliminate the overhead of creating a > connection > (IIRC ~50ms/req), which is the most annoying symptom, without incurring > the > complexity of a "real" pooler. > > They would be a win for small and medium websites that don't manage their > database transactions manually and where the complexity of maintaining an > external connection pooler isn't justified. > > Besides, when Django's transaction middelware is enabled, each request is > wrapped in a single transaction, which reserves a connection. In this > case, a > connection pooler won't perform better than persistent connections. > > Obviously, large websites should use an external pooler to multiplex their > hundreds of connections from workers into tens of connections to their > database and manage their transactions manually. I don't believe > persistent > connections to the pooler would hurt in this scenario, but if it does, it > could be optional. > > > > AFAICT there are three things to take care of before reusing a connection: > > 1) restore a pristine transaction state: transaction.rollback() should do; > > 2) reset all connection settings: the foundation was laid in #19274; > > 3) check if the connection is still alive, and re-open it otherwise: > - for psycopg2: "SELECT 1"; > - for MySQL and Oracle: connection.ping(). > > Some have argued that persistent connections tie the lifetime of databases > connections to the lifetime of workers, but it's easy to store the > creation > timestamp and re-open the connection if it exceeds a given max-age. > > So -- did I miss something? > > -- > Aymeric. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=
Re: Speeding up tests
So a few things we've done to take our test suite from 45 minutes to 12: 1. Implement global fixtures These get loaded after syncing just like initial data. Obviously this is a massive speed up as you only reload them in between transaction test cases. 2. Don't inherit from TestCase if you aren't using the db Around 10% of our tests now inherit from unit test instead. This let's us only bootstrap the db if it's required as well as skip any db flushing on those tests. 3. Speedup fixture loading More or less what is proposed by the original poster 4. Stop writing integration tests Biggest win :) learn to use mock On Jan 19, 12:47 pm, Adrian Holovaty wrote: > On Mon, Jan 16, 2012 at 10:46 AM, Anssi Kääriäinen > > wrote: > > I have been investigating what takes time in Django's test runner and > > if there is anything to do about it. The short answer is: yes, there > > is a lot of room for improvement. I managed to reduce the running > > speed of the test suite (on sqlite3) from 1700 seconds to around 500 > > seconds. On postgresql I reduced the run time from 5000 to 2500 > > seconds. > > Wow! Just wanted to say thanks for doing all of this work and making > these optimizations. I'm going to take a look at #16759, along with > your Git branch. > > Adrian -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: initial_data and post_syncdb conflicts
initial_data gets populated whenever you migrate your database (e.g.
syncdb, reset, etc). These, at least in my opinion, can be see as
default database fixtures that you should be allowed to have an
expectation of their existence. Gven that, we have *test only*
fixtures (not initial_data) that get loaded that reference (with
foreign keys) rows that are created in the initial_data load.
On Aug 19, 1:12 am, Stephen Burrows
wrote:
> Perhaps I'm missing something... initial_data is "test-only", right?
> As opposed to other fixtures which should always be loaded? Then why
> would the other fixtures rely on the initial_data?
>
> On Aug 18, 4:15 pm, David Cramer wrote:
>
>
>
>
>
>
>
> > We've been working on switching our test suite to use some new "super
> > fixtures", which are really just global, test-only initial_data style
> > fixtures. To implement this we attach to the post_syncdb, and set a
> > runonce-per-db flag (since it seems to be the only available signal),
> > but we hit some issues with the way the process flow works for flush/
> > syncdb.
>
> > Basically, all the code goes:
>
> > {flush,syncdb} => {db actions} => emit post_syncdb => load
> > initial_data
>
> > In order for our hooks to work, we had to reorder load initial_data so
> > that it happened before the syncdb signal was sent out. The reason
> > being is that some of our fixture data relied on foreignkeys which
> > were present in initial_data.
>
> > My question is, would it be reasonable to change this upstream (both
> > in syncdb and flush), or would it be better to possibly add some
> > additional signals?
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
initial_data and post_syncdb conflicts
We've been working on switching our test suite to use some new "super
fixtures", which are really just global, test-only initial_data style
fixtures. To implement this we attach to the post_syncdb, and set a
runonce-per-db flag (since it seems to be the only available signal),
but we hit some issues with the way the process flow works for flush/
syncdb.
Basically, all the code goes:
{flush,syncdb} => {db actions} => emit post_syncdb => load
initial_data
In order for our hooks to work, we had to reorder load initial_data so
that it happened before the syncdb signal was sent out. The reason
being is that some of our fixture data relied on foreignkeys which
were present in initial_data.
My question is, would it be reasonable to change this upstream (both
in syncdb and flush), or would it be better to possibly add some
additional signals?
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: required=True for BooleanFields
I'm not suggesting changing the behavior (again due to the compatibility concerns), but I completely agree with the original poster(s). Also, in my experience it's a much less common case that you're wanting an "I agree" checkbox in your form, versus a "Boolean" field which can be positive or negative. On Jun 16, 9:34 pm, Tai Lee wrote: > This has been discussed many times in the past. For better or worse, > we're stuck with the current behaviour for backwards compatibility. > > I personally think it's better this way. Without this behaviour, it > would be a PITA to implement forms that have a "required" boolean > field (must be ticked) without repeating yourself constantly by > writing custom validation for those fields. Most forms I work with > have a "Yes, I have agreed to the terms and conditions" or "Yes, I > want to receive emails from XYZ" type fields. > > If your boolean fields are not "required" (as per the definition in > Django forms, "must be ticked"), why can't you just put > `required=False` in your form? > > I'd try to avoid patching your local Django with a change like this > unless absolutely necessary so that you can cleanly upgrade and don't > end up writing code that does the opposite of what everyone else > expects. > > Cheers. > Tai. > > On Jun 17, 5:14 am, Michael Blume wrote: > > > > > > > > > In Django BooleanFields, the required flag is used to mean that the field > > must be checked for the form to validate. Required is True by default for > > all Fields, so this is the default behavior. > > > I strongly suspect that this violates principle of least surprise for most > > people including Boolean Fields in forms. It did for me. > > > I've patched it in my local Django checkout. I realize this is a > > backwards-incompatible change, so it might not be eligible for trunk any > > time soon, but FWIW, here's the patch: > > > --- i/django/forms/fields.py > > +++ w/django/forms/fields.py > > @@ -606,6 +606,11 @@ class URLField(CharField): > > class BooleanField(Field): > > widget = CheckboxInput > > > + def __init__(self, *args, **kwargs): > > + if not args and 'required' not in kwargs: > > + kwargs['required'] = False > > + return super(BooleanField, self).__init__(*args, **kwargs) > > + > > def to_python(self, value): > > """Returns a Python boolean object.""" > > # Explicitly check for the string 'False', which is what a hidden > > field -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ImportError catching in urlresolvers.py
This is currently a problem all over in the Django codebase, and I'd
love to see a generic/reusable approach at solving this everywhere.
On Jun 14, 1:19 pm, Michael Blume wrote:
> In RegexURLPattern._get_callback, we attempt to fetch the callable named by
> the URL pattern, and catch a possible ImportError if this fails. If so, we
> raise ViewDoesNotExist.
>
> try:
> self._callback = get_callable(self._callback_str)
> except ImportError, e:
> mod_name, _ = get_mod_func(self._callback_str)
> raise ViewDoesNotExist("Could not import %s. Error was: %s" %
> (mod_name, str(e)))
>
> The trouble is that the view we're importing may indeed exist, and may
> *itself* make a failed import (say of a model, or some library function), in
> which case ViewDoesNotExist becomes a badly misleading exception name, and
> the resulting traceback is cut unhelpfully short. I've noodled a bit trying
> to come up with a patch, but been stymied by the lack of information bubbled
> up by python's ImportError extension (at least in 2.6).
>
> Personally, I don't think catching ImportError/raising ViewDoesNotExist adds
> much value (we do catch it once in contrib/admindocs/views), so unless we
> can come up with a way to ensure that it only triggers when the top-level
> import fails, I'd recommend eliminating it.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Django Error Display Page
class EasyWin(object):
def process_exception(self, request, *args, **kwargs):
if not request.is_ajax(): return
impot traceback
return HttpResponse(traceback.format_exc())
On Jun 10, 1:11 pm, Daniel Watkins
wrote:
> On Thu, Jun 09, 2011 at 08:31:44PM -0700, Valentin Golev wrote:
> > What I'd really like is a stacktrace in a plain text in the html
> > commentary ("") on the very top of the page.
>
> I've openedhttps://code.djangoproject.com/ticket/16227with patch
> attached.
>
> Regards,
>
> Dan
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Test optimizations (2-5x as fast)
Here's my proposal, assuming it can be done: 1. Create default database. 2. Run a test 3. If a test has fixtures, check for, and if not, copy base table to ``name_``. 4. Start transaction 5. Run Tests 6. Roll back I think that pretty much would solve all cases, and assuming you reuse tons of fixtures it should be a huge benefit. Also if the begin/rollback aren't good enough, and we can already "copy" a database, then we could just continually copy databases each time (assuming its fast). On May 19, 6:12 am, Hanne Moa wrote: > On 18 May 2011 01:46, Erik Rose wrote: > > >> Is there a sensible to way "copy" databases in SQL? > > > SQL 2003 introduced CREATE TABLE x LIKE y for cloning the schema of a > > table. It's supported in MySQL at least. You could then do a bunch of > > INSERT INTO ... SELECTs if you deferred foreign key checks first. > > Sometimes, in order to rescue data from an overfull table (because the > cleanup-job had died and a DELETE would take too long) I've done the > following: > > - start transcation > - rename bad table > - receate the table (CREATE TABLE x LIKE would work) > - INSERT INTO ... SELECT good data into the recreated table from the > renamed table > - drop renamed table > - end transaction > > This works even when the system is up and running, on production servers. > > HM -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Test optimizations (2-5x as fast)
Is there a sensible to way "copy" databases in SQL? it's pretty obvious with things like sqlite, but outside of that seems tricky. I really like that idea, and you should definitely just be able to (at the very least) run a unique hash on the required fixtures to determine if a database is available for this or not. On May 17, 11:28 am, Erik Rose wrote: > > I would be very happy to test this against Oracle database to see is how > > much patch improves speed since previously running tests against Oracle > > has been a real pain specially all db recreate stuff took a long long > > time. > > Great! I'll post again to this thread when the patch is ready. Or, if you'd > like to try it now, you can downloadhttps://github.com/jbalogh/test-utilsand > make your test classes subclass FastFixtureTestCase rather than TestCase. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Test optimizations (2-5x as fast)
Postgres requires resetting the sequences I believe. I just assume Oracle/MSSQL are probably similar. Regarding the signals, basically we have a bunch of post_save type things, which tend to store aggregate data for certain conditions. These get populated (in some cases) in our tests, but don't correspond to a fixture or a model in the same app. -- David Cramer http://justcramer.com On Mon, May 16, 2011 at 9:09 PM, Erik Rose wrote: > Woo, thanks for the constructive suggestions! > >> Also, one thing I'm quickly noticing (I'm a bit confused why its >> setup_class and not setUpClass as well), > > I was writing to nose's hooks; didn't realize Django used unittest2 now! > >> but this wont work with >> postgres without changing the DELETE code to work like the test >> runner's TRUNCATE foo, bar; (due to foreign key constraints). > > Absolutely. I assume this is what you fix below > >> You can do something like this to handle the >> flushing: >> >> sql_list = connection.ops.sql_flush(no_style(), >> tables, connection.introspection.sequence_list()) >> for sql in sql_list: >> cursor.execute(sql) > > Brilliant! Thanks! Say, can you think of any backends in which you actually > have to reset the sequences after truncating? That seems like an interesting > decoupling to me. MySQL, anyway, does the reset implicitly; perhaps we can > optimize its sql_flush routine. > >> Unfortunately, you're still reliant that nothing was created with >> signals that uses constraints. For us this is very common, and I can't >> imagine we're an edge case there > > Can you tell me of these signals? Which ones? I don't think we use them, but > I don't want to overlook them. > > Erik -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Test optimizations (2-5x as fast)
More quick notes. You can do something like this to handle the flushing: sql_list = connection.ops.sql_flush(no_style(), tables, connection.introspection.sequence_list()) for sql in sql_list: cursor.execute(sql) Unfortunately, you're still reliant that nothing was created with signals that uses constraints. For us this is very common, and I can't imagine we're an edge case there On May 13, 9:42 pm, David Cramer wrote: > You sir, are my personal hero for the day :) > > We had also been looking at how we could speed up the fixture loading > (we were almost ready to go so far as to make one giant fixture that > just loaded at the start of the test runner). This is awesome progress > > On May 13, 4:57 pm, Erik Rose wrote: > > > > > > > > > tl;dr: I've written an alternative TestCase base class which makes > > fixture-using tests much more I/O efficient on transactional DBs, and I'd > > like to upstream it. > > > Greetings, all! This is my first django-dev post, so please be gentle. :-) > > I hack on support.mozilla.com, a fairly large Django site with about 1000 > > tests. Those tests make heavy use of fixtures and, as a result, used to > > take over 5 minutes to run. So, I spent a few days seeing if I could cut > > the amount of DB I/O needed. Ultimately, I got the run down to just over 1 > > minute, and almost all of those gains are translatable to any Django site > > running against a transactional DB. No changes to the apps themselves are > > needed. I'd love to push some of this work upstream, if there's interest > > (or even lack of opposition ;-)). > > > The speedups came from 3 main optimizations: > > > 1. Class-level fixture setup > > > Given a transaction DB, there's no reason to reload fixtures via dozens of > > SQL statements before every test. I made use of setup_class() and > > teardown_class() (yay, unittest2!) to change the flow for TestCase-using > > tests to this: > > a. Load the fixtures at the top of the class, and commit. > > b. Run a test. > > c. Roll back, returning to pristine fixtures. Go back to step b. > > d. At class teardown, figure out which tables the fixtures loaded into, > > and expressly clear out what was added. > > > Before this optimization: 302s to run the suite > > After: 97s. > > > Before: 37,583 queries > > After: 4,116 > > > On top of that, an additional 4s was saved by reusing a single connection > > rather than opening and closing them all the time, bringing the final > > number down to 93s. (We can get away with this because we're committing any > > on-cursor-initialization setup, whereas the old TestCase rolled it back.) > > > Here's the > > code:https://github.com/erikrose/test-utils/blob/master/test_utils/__init_ > > I'd love to generalize it a bit (to fall back to the old behavior with > > non-transactional backends, for example) and offer it as a patch to Django > > proper, replacing TestCase. Thoughts? > > > (If you notice that copy-and-paste of loaddata sitting off to the side in > > another module, don't fret; in the patch, that would turn into a > > refactoring of loaddata to make the computation of the fixture-referenced > > tables separately reusable.) > > > 2. Fixture grouping > > > I next observed that many test classes reused the same sets of fixtures, > > often via subclassing. After the previous optimization, our tests still > > loaded fixtures 114 times, even though there were only 11 distinct sets of > > them. So, I thought: why not write a custom testrunner that buckets the > > classes by fixture set and advises the classes that, unless they're the > > first or last in a bucket, they shouldn't bother tearing down or setting up > > the fixtures, respectively? This took the form of a custom nose plugin (we > > use nose for all our Django stuff), and it took another quarter off the > > test run: > > > Before: 97s > > After: 74s > > > Of course, test independence is still preserved. We're just factoring out > > pointlessly repeated setup. > > > I don't really have plans to upstream this unless someone calls for it, but > > I'll be making it available soon, likely as part of django-nose. > > > 3. Startup optimizations > > > At this point, it was bothering me that, just to run a single test, I had > > to wait through 15s of DB initialization (mostly auth_permissions and > > django_content_type populatio
Re: Test optimizations (2-5x as fast)
Also, one thing I'm quickly noticing (I'm a bit confused why its setup_class and not setUpClass as well), but this wont work with postgres without changing the DELETE code to work like the test runner's TRUNCATE foo, bar; (due to foreign key constraints). On May 13, 9:42 pm, David Cramer wrote: > You sir, are my personal hero for the day :) > > We had also been looking at how we could speed up the fixture loading > (we were almost ready to go so far as to make one giant fixture that > just loaded at the start of the test runner). This is awesome progress > > On May 13, 4:57 pm, Erik Rose wrote: > > > > > > > > > tl;dr: I've written an alternative TestCase base class which makes > > fixture-using tests much more I/O efficient on transactional DBs, and I'd > > like to upstream it. > > > Greetings, all! This is my first django-dev post, so please be gentle. :-) > > I hack on support.mozilla.com, a fairly large Django site with about 1000 > > tests. Those tests make heavy use of fixtures and, as a result, used to > > take over 5 minutes to run. So, I spent a few days seeing if I could cut > > the amount of DB I/O needed. Ultimately, I got the run down to just over 1 > > minute, and almost all of those gains are translatable to any Django site > > running against a transactional DB. No changes to the apps themselves are > > needed. I'd love to push some of this work upstream, if there's interest > > (or even lack of opposition ;-)). > > > The speedups came from 3 main optimizations: > > > 1. Class-level fixture setup > > > Given a transaction DB, there's no reason to reload fixtures via dozens of > > SQL statements before every test. I made use of setup_class() and > > teardown_class() (yay, unittest2!) to change the flow for TestCase-using > > tests to this: > > a. Load the fixtures at the top of the class, and commit. > > b. Run a test. > > c. Roll back, returning to pristine fixtures. Go back to step b. > > d. At class teardown, figure out which tables the fixtures loaded into, > > and expressly clear out what was added. > > > Before this optimization: 302s to run the suite > > After: 97s. > > > Before: 37,583 queries > > After: 4,116 > > > On top of that, an additional 4s was saved by reusing a single connection > > rather than opening and closing them all the time, bringing the final > > number down to 93s. (We can get away with this because we're committing any > > on-cursor-initialization setup, whereas the old TestCase rolled it back.) > > > Here's the > > code:https://github.com/erikrose/test-utils/blob/master/test_utils/__init_ > > I'd love to generalize it a bit (to fall back to the old behavior with > > non-transactional backends, for example) and offer it as a patch to Django > > proper, replacing TestCase. Thoughts? > > > (If you notice that copy-and-paste of loaddata sitting off to the side in > > another module, don't fret; in the patch, that would turn into a > > refactoring of loaddata to make the computation of the fixture-referenced > > tables separately reusable.) > > > 2. Fixture grouping > > > I next observed that many test classes reused the same sets of fixtures, > > often via subclassing. After the previous optimization, our tests still > > loaded fixtures 114 times, even though there were only 11 distinct sets of > > them. So, I thought: why not write a custom testrunner that buckets the > > classes by fixture set and advises the classes that, unless they're the > > first or last in a bucket, they shouldn't bother tearing down or setting up > > the fixtures, respectively? This took the form of a custom nose plugin (we > > use nose for all our Django stuff), and it took another quarter off the > > test run: > > > Before: 97s > > After: 74s > > > Of course, test independence is still preserved. We're just factoring out > > pointlessly repeated setup. > > > I don't really have plans to upstream this unless someone calls for it, but > > I'll be making it available soon, likely as part of django-nose. > > > 3. Startup optimizations > > > At this point, it was bothering me that, just to run a single test, I had > > to wait through 15s of DB initialization (mostly auth_permissions and > > django_content_type population)—stuff which was already perfectly valid > > from the previous test run. So, building on some work we had already done > > in this direction, I decided to skip the teardown of t
Re: Test optimizations (2-5x as fast)
You sir, are my personal hero for the day :) We had also been looking at how we could speed up the fixture loading (we were almost ready to go so far as to make one giant fixture that just loaded at the start of the test runner). This is awesome progress On May 13, 4:57 pm, Erik Rose wrote: > tl;dr: I've written an alternative TestCase base class which makes > fixture-using tests much more I/O efficient on transactional DBs, and I'd > like to upstream it. > > Greetings, all! This is my first django-dev post, so please be gentle. :-) I > hack on support.mozilla.com, a fairly large Django site with about 1000 > tests. Those tests make heavy use of fixtures and, as a result, used to take > over 5 minutes to run. So, I spent a few days seeing if I could cut the > amount of DB I/O needed. Ultimately, I got the run down to just over 1 > minute, and almost all of those gains are translatable to any Django site > running against a transactional DB. No changes to the apps themselves are > needed. I'd love to push some of this work upstream, if there's interest (or > even lack of opposition ;-)). > > The speedups came from 3 main optimizations: > > 1. Class-level fixture setup > > Given a transaction DB, there's no reason to reload fixtures via dozens of > SQL statements before every test. I made use of setup_class() and > teardown_class() (yay, unittest2!) to change the flow for TestCase-using > tests to this: > a. Load the fixtures at the top of the class, and commit. > b. Run a test. > c. Roll back, returning to pristine fixtures. Go back to step b. > d. At class teardown, figure out which tables the fixtures loaded into, > and expressly clear out what was added. > > Before this optimization: 302s to run the suite > After: 97s. > > Before: 37,583 queries > After: 4,116 > > On top of that, an additional 4s was saved by reusing a single connection > rather than opening and closing them all the time, bringing the final number > down to 93s. (We can get away with this because we're committing any > on-cursor-initialization setup, whereas the old TestCase rolled it back.) > > Here's the > code:https://github.com/erikrose/test-utils/blob/master/test_utils/__init_ > I'd love to generalize it a bit (to fall back to the old behavior with > non-transactional backends, for example) and offer it as a patch to Django > proper, replacing TestCase. Thoughts? > > (If you notice that copy-and-paste of loaddata sitting off to the side in > another module, don't fret; in the patch, that would turn into a refactoring > of loaddata to make the computation of the fixture-referenced tables > separately reusable.) > > 2. Fixture grouping > > I next observed that many test classes reused the same sets of fixtures, > often via subclassing. After the previous optimization, our tests still > loaded fixtures 114 times, even though there were only 11 distinct sets of > them. So, I thought: why not write a custom testrunner that buckets the > classes by fixture set and advises the classes that, unless they're the first > or last in a bucket, they shouldn't bother tearing down or setting up the > fixtures, respectively? This took the form of a custom nose plugin (we use > nose for all our Django stuff), and it took another quarter off the test run: > > Before: 97s > After: 74s > > Of course, test independence is still preserved. We're just factoring out > pointlessly repeated setup. > > I don't really have plans to upstream this unless someone calls for it, but > I'll be making it available soon, likely as part of django-nose. > > 3. Startup optimizations > > At this point, it was bothering me that, just to run a single test, I had to > wait through 15s of DB initialization (mostly auth_permissions and > django_content_type population)—stuff which was already perfectly valid from > the previous test run. So, building on some work we had already done in this > direction, I decided to skip the teardown of the test DB and, symmetrically, > the setup on future runs. If you make schema changes, just set an env var, > and it wipes and remakes the DB like usual. I could see pushing this into > django-nose as well, but it's got the hackiest implementation and can > potentially confuse users. I mention it for completeness. > > Before: startup time 15s > After: 3s (There's quite a wide variance due to I/O caching luck.) > > Code:https://github.com/erikrose/test-utils/commit/b95a1b7 > > If you read this far, you get a cookie! I welcome your feedback on merging > optimization #1 into core, as well as any accusations of insanity re: #2 and > #3. FWIW, everything works great without touching any of the tests on 3 of > our Django sites, totaling over 2000 tests. > > Best regards and wishes for a happy weekend, > Erik Rose > support.mozilla.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-d
Re: get_or_create can still cause IntegrityError
Would it help (in PG-world) at least if the selects where in savepoints as well? On May 7, 10:09 am, David Cramer wrote: > We hit this same issue in Postgres (it's definitely not MySQL > specific). I'm unsure of the solution or precise conditions we're > hitting it in, but I think by default we use READ COMMITTED. > > On May 7, 3:28 am, Tomasz Zielinski > wrote: > > > > > > > > > I think that get_or_create is still broken, despite this > > fix:http://code.djangoproject.com/ticket/6641 > > > The details, along with a working solution, are in my answer > > here:http://stackoverflow.com/questions/2235318/how-do-i-deal-with-this-ra... > > I'm not sure if the solution is optimal, trying to INSERT first might be > > overally more expensive, but I think that something should be done with > > get_or_create. > > > Best, > > > Tomasz -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: get_or_create can still cause IntegrityError
We hit this same issue in Postgres (it's definitely not MySQL specific). I'm unsure of the solution or precise conditions we're hitting it in, but I think by default we use READ COMMITTED. On May 7, 3:28 am, Tomasz Zielinski wrote: > I think that get_or_create is still broken, despite this > fix:http://code.djangoproject.com/ticket/6641 > > The details, along with a working solution, are in my answer > here:http://stackoverflow.com/questions/2235318/how-do-i-deal-with-this-ra... > I'm not sure if the solution is optimal, trying to INSERT first might be > overally more expensive, but I think that something should be done with > get_or_create. > > Best, > > Tomasz -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
django.conf.settings.get()
Ticket speaks for itself: http://code.djangoproject.com/ticket/10098 My vote is to reopen it, as this is very common, easy to implement, and very unlikely that it would ever need deprecated. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Expensive queryset cloning
In our profiling we've also noticed the cloning to be one of the slowest parts of the app (that and instantiation of model objects). We haven't yet, but had been planning on exploring a way to mutate the existing class in most circumstances, but haven't dug into it too much yet. On Mar 14, 11:16 pm, Alexander Schepanovski wrote: > > I'd be surprised if the cloning of querysets is actually a > > significant bottleneck relative to the database query itself > > I was too. > Query via ORM is 2-4 times slower for me than just database query + > network roundtrip. It is not only due queryset cloning, but cloning > takes 0.5-1 of that 2-4 times. > Also, I cache db queries issued via ORM, so for cache hit I need to > construct queryset but don't need to execute query. > > > unless you're doing loops with hundreds or thousands of clones, in which > > case > > it can almost certainly be optimized via Q objects. > > not loops but many ifs. I already use dicts and Q objects (when need > complex conditions) but I still have no less than 3 (filter + order_by > + slice) clones for each query. Here we see an interesting thing - > cloning querysets made me write less beautiful code. > > > If you really think it will make a significant difference for your app, > > you can probably achieve this for yourself by using your own subclass of > > QuerySet and overriding the _clone method. > > Already do this, but it does not cover User model and such. I probably > go with some monkey patch for now. > > > This is not an option. It will break quite a lot of existing code, and > > often in highly confusing ways. > > Not so much code. In most cases cloning is not needed. But it will be > confusing breakage, I agree. On other hand for an unbiased person > cloning can be confusing too. I wonder how many people were surprised > why > qs.filter(...) > does not filter qs. > > Ok, if not by default, still we can provide some .mutating() method > or .cloning(bool) method which switches queryset into/off mutating > mode. Then we can use it in some django code such as Manager.get to > avoid useless cloning. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Default project layout / directory structure
Check out django-startproject from lincolnloop.com https://github.com/lincolnloop/django-startproject Kill off all the server configs (though some of it might be cool, like Fabric integration), and I think it'd make for a pretty good base to work from if this were to go into core. On Mar 13, 9:17 pm, John Debs wrote: > I like the idea too, since I've run into a lot of situations where > some more convention here would make a decision much easier to make. > However, it isn't clear what exactly should be better defined and I > think the first step to a serious proposal would be to figure that > out. > > The only example that comes to mind is Pinax's apps/ directory for a > project's internal apps – it's a good convention that many people > already follow. My other gripe has already been taken care of, with > the integration of django-staticfiles. > > What other things did you have in mind? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Backwards Incompatible API Changes
I have no idea how I've done this, twice. On Feb 10, 2:16 pm, Alex Gaynor wrote: > On Thu, Feb 10, 2011 at 5:14 PM, David Cramer wrote: > > We're going to be deploying a backwards incompatible change to all > > "since" values in the API. Any endpoint which accepts this parameter > > to use as a form of range pagination will now include the value sent, > > as well as values before or after it (depending on the order). > > > For example, in the original implementation, since=1, would return > > results of 2, 3, 4, etc. > > > In the new implementation, since=1, would return results of 1, 2, 3, > > 4, etc. > > > We will also very shortly be implementing proper pagination using > > cursors to avoid certain race conditions when attempting to paginate. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Django developers" group. > > To post to this group, send email to django-developers@googlegroups.com. > > To unsubscribe from this group, send email to > > django-developers+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/django-developers?hl=en. > > Did this go out to the wrong mailing list? > > Alex > > -- > "I disapprove of what you say, but I will defend to the death your right to > say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Backwards Incompatible API Changes
We're going to be deploying a backwards incompatible change to all "since" values in the API. Any endpoint which accepts this parameter to use as a form of range pagination will now include the value sent, as well as values before or after it (depending on the order). For example, in the original implementation, since=1, would return results of 2, 3, 4, etc. In the new implementation, since=1, would return results of 1, 2, 3, 4, etc. We will also very shortly be implementing proper pagination using cursors to avoid certain race conditions when attempting to paginate. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Disqus API 3.0
We have been working on a new version of the API these past couple of months, and we're nearing a public release. I wanted to take this opportunity to see if any of the heavy API users (specific needs, etc) would like to chime in with what they want to see, and possible give our docs/api testing tools a whirl. Just throw your feedback here, and if you're interested in testing out the new API before it goes public, toss me an email (da...@disqus) with what your current application use and we'll go from there. Thanks! -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Changing settings per test
I was going to propose the same thing Santiago. Signals seem like the ideal candidate to solve that problem. -- David Cramer http://www.davidcramer.net On Fri, Nov 5, 2010 at 4:57 AM, Santiago Perez wrote: >> * Settings that are internally cached. For example, anything that >> modifies INSTALLED_APPS. >> >> * Settings that need to make call to reset state affected by loading >> new new group of settings. For example, if you change TZ, you need to >> call "time.tzset()" on teardown to reset the timezone. Similarly for >> deactivating translations if you change USE_I18N. >> >> * Settings that need to be removed, rather that set to None. Again, >> TZ is an example here -- there is a difference between "TZ exists and >> is None" and "TZ doesn't exist". > > Isn't it be possible to change the setattr of settings so that when ever a > setting is changed, a signal is activated and those who cache or need to > trigger actions such as time.tzset() can hook to those signals to perform > those actions? If every setting is ready to be updated then a simple > decorator should be much simpler. These hooks could even allow the owner of > a setting to raise an exception when a setting is modified if it is truly > impossible to change it at run-time. > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Changing settings per test
Agree with Alex. We're considering moving more towards decorating
views rather than class attributes. I'm not sure of the performance
implications of many classes vs decorating functions on a large class
instead, but it just seems to make more sense in some cases.
Here's a working (we're now using it) version of the previous decorator:
def with_settings(**overrides):
"""Allows you to define settings that are required for this
function to work"""
NotDefined = object()
def wrapped(func):
@wraps(func)
def _with_settings(*args, **kwargs):
_orig = {}
for k, v in overrides.iteritems():
_orig[k] = getattr(settings, k, NotDefined)
setattr(settings, k, v)
try:
func(*args, **kwargs)
finally:
for k, v in _orig.iteritems():
if v is NotDefined:
delattr(settings, k)
else:
setattr(settings, k, v)
return _with_settings
return wrapped
--
David Cramer
http://www.davidcramer.net
On Thu, Nov 4, 2010 at 2:26 PM, Alex Gaynor wrote:
> 2010/11/4 Łukasz Rekucki :
>> Funny, I had exactly the same problem today at work while refactoring
>> my application's tests suite :).
>>
>> Currently, I'm using a pair of save/restore functions: save() monkey
>> patches the settings module and returns a dictionary of old values,
>> restore() puts back the old values based on the dictionary. I usually
>> put this in setUp/tearDown so I don't have to repeat in every test. I
>> was about to propose that
>> Django's TestCase should do something similar by default.
>>
>> Both the decorator and context processor are very useful, but having
>> something to set values for the whole test case instead of a single
>> test or a block of code would be great too. I was thinking about
>> something in line of:
>>
>> class EmailTestCase(TestCase):
>> settings = dict(DEFAULT_FROM_EMAIL="webmas...@example.com")
>>
>> On 4 November 2010 21:11, David Cramer wrote:
>>> With a decorator approach here's what I whipped up:
>>>
>>> (This is dry code)
>>>
>>> def with_settings(**overrides):
>>> """Allows you to define settings that are required for this
>>> function to work"""
>>> NotDefined = object()
>>> def wrapped(func):
>>> �...@wraps(func)
>>> def _with_settings(*args, **kwargs):
>>> _orig = {}
>>> for k, v in overrides.iteritems():
>>> _orig[k] = getattr(settings, k, NotDefined)
>>>
>>> try:
>>> func(*args, **kwargs)
>>> finally:
>>> for k, v in _orig.iteritems():
>>> if v is NotDefined:
>>> delattr(settings, k)
>>> else:
>>> setattr(settings, k, v)
>>> return _with_settings
>>> return wrapped
>>>
>>> I'm not familiar with the context managers, but I imagine those would
>>> solve things like adjusting CONTEXT_PROCESSORS.
>>>
>>> On Thu, Nov 4, 2010 at 1:06 PM, Dan Fairs wrote:
>>>>
>>>>> Let me start with an example test:
>>>>>
>>>>> def test_with_awesome_setting(self):
>>>>> _orig = getattr(settings, 'AWESOME', None)
>>>>> settings.AWESOME = True
>>>>>
>>>>> # do my test
>>>>> ...
>>>>>
>>>>> settings.AWESOME = _orig
>>>>>
>>>>
>>>> Pedant: there's a small bug above which has bitten me before doing a
>>>> similar thing - settings.AWESOME ends up set to None after the test has
>>>> run if it didn't exist before.
>>>>
>>>>> Anyways, I'd love to hear how others have dealt with this and any
>>>>> other possible solutions.
>>>>
>>>> I've used Michael Foord's Mock library to patch a setting for the duration
>>>> of a test case. Chris Withers' testfixtures library also has some sugar to
>>>> provide a context manager approach, though I haven't used that in a little
>>>> while.
>>>>
>>>> Cheers,
>>>> Dan
>>>>
>>>> --
>>>> Dan Fairs | dan.fa...@gmail.com | www.fezc
Re: Changing settings per test
With a decorator approach here's what I whipped up:
(This is dry code)
def with_settings(**overrides):
"""Allows you to define settings that are required for this
function to work"""
NotDefined = object()
def wrapped(func):
@wraps(func)
def _with_settings(*args, **kwargs):
_orig = {}
for k, v in overrides.iteritems():
_orig[k] = getattr(settings, k, NotDefined)
try:
func(*args, **kwargs)
finally:
for k, v in _orig.iteritems():
if v is NotDefined:
delattr(settings, k)
else:
setattr(settings, k, v)
return _with_settings
return wrapped
I'm not familiar with the context managers, but I imagine those would
solve things like adjusting CONTEXT_PROCESSORS.
On Thu, Nov 4, 2010 at 1:06 PM, Dan Fairs wrote:
>
>> Let me start with an example test:
>>
>> def test_with_awesome_setting(self):
>> _orig = getattr(settings, 'AWESOME', None)
>> settings.AWESOME = True
>>
>> # do my test
>> ...
>>
>> settings.AWESOME = _orig
>>
>
> Pedant: there's a small bug above which has bitten me before doing a similar
> thing - settings.AWESOME ends up set to None after the test has run if it
> didn't exist before.
>
>> Anyways, I'd love to hear how others have dealt with this and any
>> other possible solutions.
>
> I've used Michael Foord's Mock library to patch a setting for the duration of
> a test case. Chris Withers' testfixtures library also has some sugar to
> provide a context manager approach, though I haven't used that in a little
> while.
>
> Cheers,
> Dan
>
> --
> Dan Fairs | dan.fa...@gmail.com | www.fezconsulting.com
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Changing settings per test
A common behavior I seem to have is the need to tweak the settings object for certain test cases. The other case is that in many cases we were (are?) relying on settings being configured a certain way for the Django tests to even work. I brought this up in #django-dev a while back, but wanted to open it up to further discussion. Let me start with an example test: def test_with_awesome_setting(self): _orig = getattr(settings, 'AWESOME', None) settings.AWESOME = True # do my test ... settings.AWESOME = _orig So the obvious problem for me here is that I'm repeating this same code flow in a lot of situations. Ignoring the fact that it's ugly, it's just not fun mangling with settings like this (at least, not fun having to reset the values). My proposal is to include some kind of utility within the test suite which would make this easier. There's a couple ways I could see this working: 1. The settings object could be copied and reset on each case. 2. The settings object could be replaced with a Proxy which stores a copy of any value changed since reset, and returns that value if present. It could then simply just be reset (clear the proxy's dict) on each setUp. Anyways, I'd love to hear how others have dealt with this and any other possible solutions. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Could not import man.credit.views, python 2.6.5. "No module named dl" error
It's an import issue with your app unrelated to django core. Someone here will eventually tell you to direct that to django-users, so you might as well hop over there now. On Jul 12, 4:31 pm, pacman wrote: > Hi, > > I've upgraded our python from 2.4 to 2.6.5. When connecting to django > server (1.2.1), it shows the following error: > > I assume it's still searching for deprecated dlmodule.so (py 2.4) or > dl.so (py 2.6). We didn't compile that module in py 2.6.5. Anyone has > seen this error and knows fix on the django side? Thanks! > > TemplateSyntaxError at / > Caught ViewDoesNotExist while rendering: Could not import > man.credit.views. Error was: No module named dlRequest Method: GET > Request URL:http://devpmmtg1:9060/ > Django Version: 1.2.1 > Exception Type: TemplateSyntaxError > Exception Value: Caught ViewDoesNotExist while rendering: Could not > import man.credit.views. Error was: No module named dl > Exception Location: /usr/local/python-2.6.5/lib/python2.6/site- > packages/django/core/urlresolvers.py in _get_callback, line 132 > Python Executable: /appl/pm/vendor/python/lx-x86/python-2.6.5/bin/ > python > Python Version: 2.6.5 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Cross-field Model Validation and Ticket #13100 (Was: Re: High Level Discussion about the Future of Django)
Realizing my original statement I was regarding this thread, in this thread, it's obvious that this has gone completely off track. I might have to take back everything I thought about this being useful. If you want to address a SPECIFIC concern, it makes sense to do that under its own topic. Think of this mailing list like a forum, as, after all, many of us browse it just like one. When a "thread" happens to have 12 different topics it loses its value fast. On Apr 19, 3:44 pm, Richard Laager wrote: > In the end, *my* requirement is that I have *some place* to put > validation code that 1) can see the whole model instance, 2) will be run > from the admin interface, and 3) will return nice validation failures to > the user (not throw exceptions that will give the user a 500 error and > send me an email). > > A) Is this an unreasonable pony? If so, why? > B) If not, how can I implement this such that it will get accepted? > > I'd like to have it in for 1.2 if possible, as the model validation > interfaces are new. Once released, there will be more > backwards-compatibility guarantees to maintain. > > > > > > On Mon, 2010-04-19 at 10:53 -0500, James Bennett wrote: > > On Mon, Apr 19, 2010 at 10:16 AM, Richard Laager wrote: > > > On Mon, 2010-04-19 at 07:55 -0700, orokusaki wrote: > > >> With all respect, you still haven't addressed my main concern: You > > >> told me that it was because of backward compatibility that this simple > > >> change couldn't be put in the trunk. It is backward compatible. If I'm > > >> wrong, it would suffice to have a simple explanation of what it > > >> breaks. > > > > I'd like to second this question. orokusaki suggested a couple of things > > > in ticket #13100, but I'm seconding specifically this comment: > > >http://code.djangoproject.com/ticket/13100#comment:8 > > > The difference between how ModelForm works and how Model works is > > that, if you're overriding clean() on a ModelForm subclass, you don't > > automatically get uniqueness validation -- you have to call up to the > > parent clean(), or manually apply the uniqueness validation in your > > own clean(). > > Thank you for this explanation. > > orokusaki noted in ticket #13100: > "The only difference between its implementation and > ModelForm?._post_clean() is the internal check it makes before running > validate_unique()." > > So is the actual issue here that naively calling Model.full_clean() will > always run Model.validate_unique(), when the existing > ModelForm._post_clean() code only calls Model.validate_unique() when > self._validate_unique? > > If so, Model.full_clean() is new in 1.2. So, could we just add a kwarg > like this (or similar)? > def full_clean(self, exclude=None, validate_unique=True) > > Then, it would be modified to only call Model.validate_unique if the > validate_unique argument was True. > > Then, you could call Model.full_clean() from ModelForm._post_clean(), > passing self._validate_unique and preserve the same behavior. > > Alternatively, could we add another function to Model that allows for > whole-model validation but does not call Model.validate_unique() and > then call that from ModelForm._post_clean() instead of calling > Model.full_clean()? Of course, Model.full_clean() would have to call > this new validation function as well. > > Richard > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/django-developers?hl=en. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: High Level Discussion about the Future of Django
I just want to throw my 2 cents into the ring here. I'm not against a fork, but at the same time I want to see the Django mainline progress. However, let me tell you my story, and how I've seen the Django development process over the years. I started with Django 4 years ago. It was cool, shiny, and let us get up and running. At the time, were were one of the largest Django installations. We had needs, and some of those needs were met. However, many were not. We were the only ones furiously trying to get the core team to accept our patches, ideas, etc.. Now while I'm not going to say every idea we had was right for Django, there were in fact many that were great, and eventually have made it into trunk. There are also still many (lets take the classic #17 here), that still haven't made it into trunk, even though people have been even more aggressively pushing them lately. I honestly can only remember a single patch that I've committed that has ever been fully integrated into trunk (select_related refactor). Now, the development process has changed with Django over the years, but I will sadly say that I feel it's been for the worst. I've completely given up on trac, submitting patches, or even participating. Now while some may not like my aggressive tacts (e.g. James) that doesn't mean what I've brought to the table hasn't been needed. For the last year or two all I've seen on the trac whenever I took up the time to write a patch, or even submit a ticket, was a closure of "wontfix" for some, honestly, stupid reason. It just isnt worth my time to submit the same ticket 3 times just so its "correct", or it fits everyones needs. Tickets are not patches, and they shouldn't be treated like "if this one isnt accepted, create a new one". I think there's a split within the Django core team right now and I strongly believe that unless you can tirelessly convince a core developer (no matter how large the following), a feature is not going to make it into mainline. This to me is a serious issue when you're talking about an open source, community contributed project. Sure, the core team does a large amount of the work, but not without help from the community. I'll take this back to my old analogy, not everyone is building a blog, and if they are, they can go use WordPress. Many, many things have gone ignored for far too long. I love Malcom's ORM refactor, but that was at a standstill for I don't know how long, and that entire time any patch which was related to improvements to the ORM was ignored simply stating "we're working on a refactor of the ORM". This philosophy seems to continue still today. Just recently there was a post about "High Level Talk About Django" (or whatever it was called). Now while the thread didn't make a whole lot of sense in general, it was just an attempt to gather some ideas, and brainstorm. Immediately it was shut down by the core developers. What frustrates me even more is all of this pony talk. If there's one thing I dislike it's Django's philosophy that "if it can be done 3rd party, do it", yet even the simplest things, like the template engine, have better 3rd party implementations (Jinja2). Django still doesn't have migrations. It still doesn't have dependancies. It's seriously lacking in many areas which other (albeit lesser) alternatives such as Rails have made available for far too long. Now while there's great 3rd party apps for things like this (South), and there's a few mediocre sites to find pieces of code (Django Snippets), this doesn't solve the problem which is really going on in Django: The community cant contribute beyond what the core team deems necessary. For me, I've entirely given up on trying to give back to Django. I've written enormous amounts of questionable code simply so I didn't have to patch Django, or even bother dealing w/ the process of Django's development. Monkey patching, ugly metaclass hacks, you name it. Anything that's made it easier to avoid this "process", has made it easier for us to develop. I continue to build these "ponies", but that doesn't make them any easier to integrate in Django. All in all, I think some things have been ignored for far too long. Simple things, again, like migrations, JSON and RESTful utilities, and even the tools to make development easier (the debug toolbar hasn't been around that long). Yet so much time is spent on things like refactoring the admin (while it's useful, in the big picture, its not flexible, and never can be), the template system, and many other things which have been done and done again by other people. Again, this is just my opinion, and I do know that many share it. This has been one of the largest outcries of people I've seen in a while. I honestly can't see that I see a fork succeeding, but I would definitely like to see what can happen to make the "process" friendlier to people like myself and some of the others who have posted here. Really, for me, I just don't (nor do I want to) have the time to
Re: Front-End Developer - Contract/Telecommute | 40-50/hour
On Apr 7, 4:47 pm, OSS wrote: > Front-End Developer - Contract/Telecommute | 40-50/hour > > My client is a B2B media company and they are looking to hire a Front- > End Web Developer for a few upcoming projects, ranging from an online > publication development project to social media applications. This is > contract work for now, but it might have the potential to turn into a > permanent position for the right person. > > Here's a quick run down of the important stuff they are looking for, > ideally: > > * Solid JavaScript development experience. JQuery is a huge plus. > * Solid PHP experience. WordPress plug-in development would be great. > * Python experience would be a huge plus. Django experience an even > bigger plus. > * Solid HTML/CSS. This almost goes without saying. > * Experience working with templating languages would be great. > * Strong visual design skills. You should be able to craft amazing- > looking, functional user-interfaces even if you're not working off an > explicit mockup a designer has given you. > * The ability to learn and adapt quickly to new languages, APIs, > development requirements, etc. > > Above all else, what they require is a strong commitment to > productivity. They are not hiring somebody because their team lacks > the skill sets mentioned above; they are hiring somebody because they > have a lot of projects on their plate and they need to increase their > output. They are only interested in working with people who are > passionate about helping them increase the quality and quantity of > development work they do. > > Although my client has an office in NYC, your geographic U.S. location > is not important, since the development team is scattered around the > country. Wherever you live, you'll have to be comfortable working > with others remotely. You must be able to contribute as little as 20 > or as many as 40+ hours per week. > > To be considered, please submit your RESUME and HOURLY requirements to > beau[AT]open-source-staffing.com > > Thank you, > Beau J. Gould > -- > Open Source > Staffinghttp://www.open-source-staffing.comhttp://www.facebook.com/beau.gould > beau[AT]open-source-staffing.com -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: proposal: ask a queryset to not raise exceptions
I definitely would like to see this handled in Django, but not in the
way mentioned. I personally think there does not need to be an option
for what it raises. I think of this in the situations where find
methods return either a -1, or an Exception, based on which method you
call.
Now I can't come up w/ any reasonable alternatives, and a param
to .get() is bad simply due to the fact that i'd reserve a column
name, but something like
Model.objects.get() -- raises DoesNotExist
Model.objects.get_when() -- returns None
On Mar 7, 4:14 pm, Vitaliy Kulchevych wrote:
> http://code.djangoproject.com/ticket/5741
>
>
>
>
>
> On Sun, Mar 7, 2010 at 11:48 PM, Chris wrote:
> > In my projects, I use this pattern a lot:
>
> > try:
> > obj = SomeModel.objects.filter(**crazy_filters_here).latest()
> > except SomeModel.DoesNotExist:
> > obj = None
>
> > Basically, I throw a filter at a queryset and if some value comes out,
> > I use it, If the filter happens to filter out all rows, I want the
> > queryset to return None. The way it works now is kind of tedious. I
> > can understand in some instances a DoesNotExist exception is whats
> > best, but other times, it would be best to just return None.
>
> > This I think would be better:
>
> > obj =
> > SomeModel.objects.filter(**crazy_filters_here).empty(raise=None).latest()
>
> > if you want the old behavior:
>
> > obj =
> > SomeModel.objects.filter(**crazy_filters_here).empty(raise="DoesNotExist").
> > latest()
>
> > Also, instead of:
>
> > from django.shortcuts import get_object_or_404
> > obj = get_object_or_404(SomeModel, **crazy_filters_here).latest()
>
> > you could do:
>
> > obj =
> > SomeModel.objects.filter(**crazy_filters_here).empty(raise="404").latest()
>
> > Another pattern that appears in my code is this:
>
> > try:
> > val =
> > SomeModel.objects.filter(**crazy_filters_here).values_list('val',
> > flat=True)[0]
> > except IndexError:
> > val = None
>
> > It should also allow this to work without raising an IndexError:
>
> > val =
> > SomeModel.objects.filter(**crazy_filters_here).empty(raise=None).values_lis
> > t('val',
> > flat=True)[0]
>
> > because empty() should return a NoneQuerySet or FourOhFourQuerySet or
> > something like that, which allows values() and slicing, but are
> > ignored.
>
> > I don't know, this is just an idea I had, what do you all think?
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To post to this group, send email to django-develop...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-developers+unsubscr...@googlegroups.com.
> > For more options, visit this group
> > athttp://groups.google.com/group/django-developers?hl=en.
>
> --
> live free or die;
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: examples directory
It's already been done orokusaki. The examples were (humbly) horrible as well. No template usage, just generic HttpResponse. That's basic Python, and docs > examples (in code). On Feb 15, 11:52 pm, orokusaki wrote: > -1 I think examples, broken or working, are very helpful for absolute > beginners. Maybe there should be strict warnings about the quality of > the code (in a similar fashion to the ones that warn you when you view > old docs). -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: QuerySet.exists() - Possible bug?
MySQL, in this situation, would have to actually select a row to
return a result, so it's slower. If it was just select 1 as a from
table where indexed_value = N, it doesn't even hit the tables, just
the indexes.
It's definitely not more efficient, and probably just an oversight
somewhere.
On Feb 10, 7:35 am, Jeremy Dunck wrote:
> On Wed, Feb 10, 2010 at 2:05 AM, Karen Tracey wrote:
>
> ...
>
> >> * What version of Django (including SVN revision, if appropriate) are
> >> you using?
>
> > I tried current trunk and backed off to the changeset where the function was
> > added -- query was the same. Looking at the idiom it replaced, the extra
> > select of the constant 1 value as 'a' was followed by a values('a') that
> > turned the result into a ValuesQuerySet. I don't see where this is being
> > done with the exists() implementation. As a result it seems to be a regular
> > QuerySet with a query with default_cols=True, resulting in all the default
> > columns getting pulled into the query.
>
> It seems odd to me that adding columns to the result of a returned row
> would be significantly slower than the PK. It also seems to me that
> using the default ordering might be unnecessary overhead, too.
>
> But I wonder if different DBs return different things based on the
> columns in the select clause? Perhaps Malcolm had a reason for making
> the change. :-/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
The first three have been huges ones with us. We're just now running into the settings issue, but would love to see what people can come up with for solutions (we dont have a good one). Glad to see multi db is finally shipping, and excited to see what can be done for startup procs. On Jan 19, 3:26 pm, Jacob Kaplan-Moss wrote: > Hi folks -- > > I had a conversation this morning with one of my clients. In the > interests of being a good corporate citizen I'll refrain from > mentioning who other than (a) they're big (Fortune 1000), (b) you've > heard of them, and (c) they're using Django. > > Before our chat, they'd invited any engineers internally to bitch > about Django, and they shared some of the pain points they'd come > across. I took some notes; the really good news is that it's a short > list, and we already know about most of it. > > Before I share the list, I should mention that I'm not arguing we have > to immediately devote resources to these problems, nor even that we > have to solve them. I'm sharing this just as food for thought, and as > a perspective from a class of developers who mostly don't participate > in our process at all. > > So: > > The main issue they have is multiple database support. Music to my > ears (and especially Alex's :) since it looks like 1.2 will solve most > (all?) of their issues. My main takeaway from this point was that the > best way to please companies like this one will be to ship 1.2 on > time. Which will happen. So we rock. > > The next big one was the "startup signal" issue -- they've got lots of > code that needs to run at startup time, and there's no great mechanism > to do that currently. The core devs have talked about this one a *lot* > over the years, and it's obviously a hard one -- for one, there's no > clear definition of what "startup" means -- but a common theme seems > to be that bigger, more complex applications need some way to cleanly > run one-time, expensive startup tasks. > > Next, we discussed the difficulty of maintaining large installations > with multiple sites, a plethora of apps, and unspeakable possible > combinations thereof. Django's settings mechanism is simple and easy > to use, but once you have hundreds of sites it starts getting really > tricky to keep things in sync across hundreds of settings files. We > talked at some length about different possibilities for dynamic > settings infrastructure; this would especially come in handy for folks > wanting to build application servers/services like App Engine or the > nascent Toppcloud. > > If you've not yet checked out Toppcloud, do so now. I'll > wait. > > Finally, we ruminated over the difficulties in building rich internet > applications. Sure, writing HTML/CSS/JS/Python/SQL by hand works fine, > but we doesn't really have a good answer for the people who want > something IDE or GUI-ish. Meanwhile, Adobe and Microsoft are putting > all sorts of marketing dollars into Flex/Silverlight, and although > HTML5 can do some amazing things, the lack of tooling is a big danger. > (I've written at more length about this in the > past:http://jacobian.org/writing/snakes-on-the-web/#s-rich-web-applications). > > Of course, there may not be much us backend folks can do about this > problem -- I'm certainly not enough of a GUI application guy to be > able to even start to think about this problem -- but the lack of an > end-to-end solution will nonetheless put some off from writing web > applications with open source tools. > > So there we are. This is very much a brain dump, and I don't really > expect any concrete action to result from it. However, I found some > really interesting stuff there, and I thought I'd share. > > Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Model validation incompatibility with existing Django idioms
For us we definitely use this behavior, and I'm guessing this is about to bite us in the ass. I would think a simple fix would be to have a commit=False, and validate=True keyword arg. By default, validate is NoInput, but if commit is False it defaults to False. Wouldn't that be a simple enough backwards compatible workaround? On Jan 7, 10:40 am, Honza Král wrote: > Hi everybody, sorry for the late reply, was busy. > > Just to add few points that lead us to do it this way: > > ModelForm has a save() method that saves the model. It is reasonable > to assume that if the form is valid, the save() method call should > succeed, that's why the entire model is validated. > > We could create a PartialModelForm, that would have save() method only > when editing (and not adding) models and have other method (or > enforced commit=False) for retrieval of the partial model. This form > would only call validation on the modelfields that are part of the > form (not being excluded). This is the behavior I understand everybody > expects from ModelForm, so, for backwards compatibility, we could call > it just ModelForm. > > Also please mind that it could prove difficult to do half the > validation in one place and other fields elsewhere without a form. > Models, as opposed to forms, don't have a state in sense of > validated/raw and they don't track changes to their fields' data. > That's why validation is run every time and the results are not cached > on the instance. > > Few quick questions to get some ideas: > > 1) If editing a model (passed an instance parameter to __init__), even > with a partial form, do you expect model.full_validate being called? > > 2) Are you OK with ripping save(commit=False) functionality to some > other method? (current functionality can stay with a deprecation > warning for example) > > 3) Would you want errors raised in model validation after it being > created by a form (commit=False) to appear on the form? > > on subject of inlines: > 1) Is it acceptable to create a model and not it's inlines in the > admin if the modelform validates but the inlines don't? > > 2) How can we limit people's validation code to avoid validating FKs > in inlines since users can (and should) create their own validation > logic on Models? Especially when we try and treat admin as "just a > cool app" and not something people should really design for. > > 3) How would you feel on creating an option to enable behavior (1) ) > and document that models with side effects in their save and delete > should really go for that? > > 4) Making 3) the default and enable switching it off if people want > their admin page to save atomically. > > Please let me know what you think > > Thanks! > > Honza Král > E-Mail: honza.k...@gmail.com > Phone: +420 606 678585 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Session/cookie based messages (#4604)
I'm with Luke on this for the exact reasons he describes. Sent from my iPhone On Dec 5, 2009, at 7:24 PM, Russell Keith-Magee wrote: > On Sun, Dec 6, 2009 at 10:28 AM, Luke Plant > wrote: >> On Sunday 06 December 2009 00:56:56 Russell Keith-Magee wrote: >> Really? Files definitely seem to be more on the "storage" side of things: http://code.djangoproject.com/browser/django/trunk/django/core/fi les/storage.py >>> >>> The problem we have here is that we have all sorts of >>> inconsistencies introduced by history. Yes, files and templates >>> use function/class level specifiers, but db, session, cache, and >>> email all use module level specifiers. Module level specifiers >>> have the majority at this point, and personally, I prefer the >>> module approach - it shortens the user configuration strings, and >>> it enforces good usage of modules. >> >> I would have to question "enforcing good usage of modules"! It >> enforces a *single* way of using modules, that is, one class per >> module. > > The fact that it is one class per module is, IMHO, incidental. It's > one *concept* per module - one *backend* per module that I consider to > be a design pattern worth enforcing. > >> That is not necessarily 'good', it certainly isn't Pythonic >> in my experience, and it could well be bad. As such, it would be >> opinionated in a bad way - it forces other developers to have a >> proliferation of modules which might be very inconvenient. Why not >> just leave the decision in the hands of the people who are writing >> and >> organizing the code? We are all consenting adults here. > > The only time I can see this as inconvenient is if you want to put two > backends in the same module - to which I say you should have two > modules anyway. > > A part of my like for modules is the aesthetic of the end settings: > > BACKEND = 'path.to.module.backends.cookie.CookieBackend' > > vs > > BACKEND = 'path.to.module.backends.cookie' > > I vastly prefer the latter for the duplication it avoids. > >> In addition to the arguments in my other message, you also have the >> case where some third party writes a backend, and then realises they >> need another. So, they had: >> >> acme.messages.MessageStorage >> >> and just want to add >> >> acme.messages.SuperMessageStorage >> >> Instead, they are forced to move code around so each of these can >> live >> in its own module, or else have stub modules that serve no purpose >> except to appease the Django-gods. > > Again, I call this good code modularity. It may well be that the > second module requires nothing but an import and a configuration of > the first, but I would still argue that it would be good practice to > use two modules. > >> I agree that it would be good to be consistent if possible, but I >> would also like to see the better convention used, and the e-mail >> system could just as easily be changed as the messages system at this >> point, which would swing the majority the other way (it's actually >> currently "4-all" by my count, and that doesn't include the other >> configurable callables and classes, such as TEST_RUNNER, >> CSRF_FAILURE_VIEW, context processors, middlewares and view >> functions, >> which all use full paths). And #6262 could also go the other way too. > > I'm not sure I pay the comparison with middlewares, context processors > and the like. A middleware is, by definition, just a class. Off the > top of my head, I can't think of any examples where a middleware > module is split into submodules - most apps just define a middleware > module and put all relevant middlewares in that module. Hence, there > is no namespace duplication. > > On the other hand, backends are consistently broken into submodules, > because they're a lot more complex. The backend isn't defined by a > single class - conceptually, there's can be multiple classes, support > data structures, etc. The database backends are the best demonstration > of this, but IMHO the broader concept exisis and is worth preserving > in other backends. > > However, I'm also willing to admit that personal preference is a > factor here. We may just need to push this up for a BDFL judgement. I > would certainly prefer module level definitions, but at the end of the > day, I don't think I'd lose much sleep if the decision went the other > way. > > Yours, > Russ Magee %-) > > -- > > You received this message because you are subscribed to the Google > Groups "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com > . > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en > . > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send
Re: Session/cookie based messages (#4604)
I agree, this is 30 minutes of work to change the usage in Django, and it should be done with the inclusion of the messages patch. David Cramer On Fri, Oct 16, 2009 at 1:08 PM, Tobias McNulty wrote: > > On Fri, Oct 16, 2009 at 5:10 AM, Luke Plant wrote: > > I think this means that either the deprecation cycle would have to > > pushed back one (i.e. pending deprecation warning in 1.3, deprecation > > in 1.4, removed in 1.5), or core/contrib should be fixed by 1.2. I > > would strongly prefer the latter, and this would affect my vote: I > > don't want two messaging systems in Django, and if user messages are > > not deprecated, then we do have two systems. > > I agree that using deprecated code in the core is setting a bad > example. A quick review of the code shows 8 calls to > message_set.create: 3 in auth, 2 in admin, and 3 in the > create/update/delete family of generic views. This definitely sounds > to me like a manageable update for 1.2. > > Per feedback from Jacob, Chris, and Luke, I updated the notes on the > existing API, the transition plan, and the potential API on the wiki: > > http://code.djangoproject.com/wiki/SessionMessages > > Cheers, > Tobias > > > -- > Tobias McNulty > Caktus Consulting Group, LLC > P.O. Box 1454 > Carrboro, NC 27510 > (919) 951-0052 > http://www.caktusgroup.com > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Session/cookie based messages (#4604)
I don't agree with one calling doing logging and notices. There's really no reason to mix in logging with the notices framework as they serve completely different purposes. I believe Russel is just suggesting the APIs match so that there is no additional learning curve, which makes complete sense. ---- David Cramer On Tue, Oct 13, 2009 at 2:53 AM, Hanne Moa wrote: > > On Tue, Oct 13, 2009 at 09:27, Russell Keith-Magee > wrote: > > I'm just > > noting that adding Django support for Python logging is also on the > > cards for v1.2, and it seems weird that we would introduce two APIs > > with similar external APIs, but not try to maintain parity. > > In fact, it would be very useful to both log and add a message at the > same time, iff there is an error. So that shouldn't deliberately be > made hard to do but maybe suitable for a lazy man's shortcut: one call > that does both. > > > HM > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Session/cookie based messages (#4604)
Sorry for the misunderstanding, I was talking specifically on retrieval of
messages. I definitely see no reason not to keep it in line with the logging
module in terms of errors/warnings/info messages. However, using things to
"retrieve all error messages" should be left up to the user. It's no quicker
doing it internally than it is using itertools.groupby or regroup in a
template.
----
David Cramer
On Mon, Oct 12, 2009 at 1:32 PM, Tobias McNulty wrote:
>
> On Mon, Oct 12, 2009 at 2:02 PM, David Cramer wrote:
> > I'm -1 on adding .errors or whatever. The main reasoning being that
> "levels"
> > are simply numeric values (even though django-notices does provide
> default
> > labels for its built-ins). Regroup is very easy and I don't think this is
> > something that needs built-in, as there's no better way to optimize it
> than
> > regroup. However, with doing this it'd be very important that it doesn't
> > clear the messages unless you're pulling it out. django-notices handles
> this
> > by popping out elements in the iter (I believe), so that if you don't pop
> > the message out, it's still present.
>
> In Python logging the levels are integers, but you still get
> convenience methods like logger.error('msg'), etc.
>
> I don't see why a similar method to filter the messages by type would
> be a bad thing.
>
> For the record I have no personal need for this feature (and hence
> strong feelings about it), but nor do I see why adding it would be a
> bad thing IF the need is actually there. Right now it sounds like
> regroup will work fine for its original proponent, so, in the interest
> of keeping things simple, let's assume it will not be included unless
> someone brings up a compelling case.
>
> Tobias
> --
> Tobias McNulty
> Caktus Consulting Group, LLC
> P.O. Box 1454
> Carrboro, NC 27510
> (919) 951-0052
> http://www.caktusgroup.com
>
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Session/cookie based messages (#4604)
I'm -1 on adding .errors or whatever. The main reasoning being that "levels"
are simply numeric values (even though django-notices does provide default
labels for its built-ins). Regroup is very easy and I don't think this is
something that needs built-in, as there's no better way to optimize it than
regroup. However, with doing this it'd be very important that it doesn't
clear the messages unless you're pulling it out. django-notices handles this
by popping out elements in the iter (I believe), so that if you don't pop
the message out, it's still present.
David Cramer
On Mon, Oct 12, 2009 at 11:19 AM, Paul McLanahan wrote:
>
> On Mon, Oct 12, 2009 at 12:11 PM, Tobias McNulty
> wrote:
> > I have no particular issue with allowing messages to be iterated in
> > full and/or by type. We could also just sort the messages and let you
> > use {% regroup messages by tag %}, but I suppose you want the ability
> > to pull out a particular type of error and that could get tedious with
> > {% regroup %}.
>
> I think regroup would be fine. My thought was that with iteration by
> type you'd be able to easily highlight just one type if you so choose,
> and even have any of the types displayed in a completely different
> area of the markup if you needed. Having both abilities would be the
> best-of-all-possible-worlds in my opinion, but this is starting to
> smell like scope-creep. I'll just say that the backends and API are
> the most important things, and having individually iterable lists is
> just a "nice to have." I'd definitely be happy to help with the
> implementation though if the group thinks it a worthy feature.
>
> Paul
>
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Session/cookie based messages (#4604)
I would definitely say it needs to be configurable then at the least. As using cookies to send messges is fine if you guarantee the entity is removed onthe next hit. Otherwise it becomes a performance issue storing them for longer periods of time. While I don't think that is common it should definitely be a factor in the decision. Sent from my iPhone On Oct 12, 2009, at 9:39 AM, Tobias McNulty wrote: > > On Mon, Oct 12, 2009 at 10:21 AM, David Cramer > wrote: >> I also don't think this problem is being addressed here. Yes you >> could >> pass messages to the context, but you would lose the ability to >> retrieve those variably. I believe storing it in the existing session >> is the best appoach for django's builtin support. > > I'm not exactly sure what problem you see as not being addressed, and > I don't think you were arguing against this, but it has been noted > elsewhere that it's more desirable to store the messages directly in a > cookie, if possible, to avoid the extra database or cache hit. > > It seems well worth it to me and there are solutions out there that > try to store the messages in a cookie and then fall back to the > session for longer (>4kb) messages. > > Tobias > -- > Tobias McNulty > Caktus Consulting Group, LLC > P.O. Box 1454 > Carrboro, NC 27510 > (919) 951-0052 > http://www.caktusgroup.com > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Session/cookie based messages (#4604)
I also don't think this problem is being addressed here. Yes you could pass messages to the context, but you would lose the ability to retrieve those variably. I believe storing it in the existing session is the best appoach for django's builtin support. Sent from my iPhone On Oct 10, 2009, at 8:00 PM, Tobias McNulty wrote: > > On Sat, Oct 10, 2009 at 7:03 PM, veena wrote: >> Today I was on very bad wifi connection. Constantly dropped. 20 >> seconds to load a page. >> I save in admin in two tabs and got a notice in one tab from the >> other >> tab. >> >> But I agree, I defer this "bug" for the first release od django >> messaging. I think django isn't now in right mood to add there some >> functionality like adding of query parameters to response object by >> some application. Maybe in future. > > AFAIK this will become a non-issue with the advent of HTML5, as each > window/tab will have its own session. > > Tobias > -- > Tobias McNulty > Caktus Consulting Group, LLC > P.O. Box 1454 > Carrboro, NC 27510 > (919) 951-0052 > http://www.caktusgroup.com > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Session/cookie based messages (#4604)
The proposal per your email is more or less how django-notices works.
Sent from my iPhone
On Oct 10, 2009, at 12:53 PM, Tobias McNulty
wrote:
>
> On Sat, Oct 10, 2009 at 1:19 PM, Tobias McNulty > wrote:
>> Things that still need to be discussed/done:
>>
>> * Coming to consensus on what 3rd party app we actually choose to
>> extend/modify to fit into Django
>>
>> * What to do with the existing user message API (see
>> http://code.djangoproject.com/wiki/SessionMessages#IntegratingwiththeExistingAPI
>> )
>>
>> * Review/add to the TODO list on the wiki
>> (http://code.djangoproject.com/wiki/SessionMessages#TODOOnceaSolutionisChosen
>> )
>
> I should have also added:
>
> * Coming to consensus on a de facto standard API suitable for
> inclusion in Django
>
> I originally put it on the TODO list, but in retrospect the discussion
> needn't wait till we pick a solution.
>
> As a practical starting point, I copied the API for django-notify to
> the wiki page. I like the general form of that API, but I'd be
> slightly more happy with something like:
>
> from django.contrib import messages
>
> request.messages.add('message', messages.INFO)
> # or
> request.messages.add('message', classes=(messages.WARNING,))
> # or
> request.messages.error('message')
>
> A la python logging, I think endorsing a specific set of message
> classes or tags (but still allowing them to be extended) is A Good
> Thing because it helps reusable apps provide more customized feedback
> in a standard way.
>
> Tobias
> --
> Tobias McNulty
> Caktus Consulting Group, LLC
> P.O. Box 1454
> Carrboro, NC 27510
> (919) 951-0052
> http://www.caktusgroup.com
>
> >
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Let's Talk About ``to_field``
I believe someone had linked a ticket before, but I was unable to find one, so I went ahead and submitted it here: http://code.djangoproject.com/ticket/11938 David Cramer On Wed, Sep 23, 2009 at 6:41 PM, David Cramer wrote: > As usual, my apologies for lacking context :) > > The problem happens when you try to query with a model containing a foreign > key that references a to_field. It fails to pass the proper attribute. > > # Exampe 1, a simple get lookup > class ExampleBrokenModel(models.Model): > user = models.ForeignKey(User, to_field="username") > > user = User.objects.get(username="a_username") > > # The following will fail due to the fact it's going to try to do user_id= > user.pk, rather > # than user_id=user.username > ExampleBrokenModel.objects.get(user=user) > > I will attach additional tests to a ticket, see also [1] > > I'm not aware of this ever working properly, but I'm not going to dig up > tickets. You can take my word for it as we've been dealing with it at this > project for a year, and had been working with Django almost 2 years before > that. Either way, it's not a big deal, the fact is it doesn't work as > advertised, and it is advertised. > > You would have to talk to the author of newforms-admin, which I believe was > Brian Rosner. There is some special code for handling to_field lookups in > the admin. > > [1] http://www.pastethat.com/LUYWh > > > David Cramer > > > > > On Wed, Sep 23, 2009 at 6:27 PM, Karen Tracey wrote: > >> On Wed, Sep 23, 2009 at 4:59 PM, David Cramer wrote: >> >>> >>> I haven't brought this up for quite a while, but ``to_field`` still >>> does not function properly (at least as of 1.1, I haven't noticed any >>> changes addressing it though). The issue is that anywhere you do a GET >>> lookup, it doesn't process the to_field properly:: >>> >>># TODO: waiting on to_field fix >>>try: >>>opt = cls.objects.get(business=business.business_id, >>> key=key) >>>except cls.DoesNotExist: >>>opt = cls.objects.create(business=business, key=key, >>> value=value) >>> >>> >> That's supposed to illustrate the problem with to_field? It's a snippet >> of code lacking any context with a cryptic TODO comment. What is the model >> definition here (which would at least give a clue where to_field applies)? >> How would this code be different if this "to_field fix" were applied? >> >> This query: >> >> >> http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&summary=~to_field&order=priority<http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&summary=%7Eto_field&order=priority> >> >> shows 4 open tickets with the "to_field" in their summary. Is the problem >> you are referring to one of these? >> >> >>> This seems to have been solved in the admin branch when newforms-admin >>> landed, but seems mostly ignored in the core field's. This issue has >>> also existed as far as I can remember, which is quite some time :) >>> >>> >> I've not been around as long as you so without reference to a ticket or >> something unfortunately I have no idea what problem you are talking about. >> I also don't know what you mean by it being fixed in admin but not in "core >> fields". Admin doesn't define fields, so huh? I can't really parse what >> you are trying to say here. >> >> >> >>> So, what do we need to get the ball rolling on this. I don't have much >>> time myself to commit, but if it's minor things such as writing some >>> tests that easily reproduce all errors we've found. We'll glady commit >>> those. Other than though my day is just filled, but I'd love to see >>> this get fixed. It's something that's advertised as a core feature, >>> has been for 3+years, and simply doesn't work. >>> >>> >> The tests you mention filed in a ticket, or attached to one of those >> tickets from above if indeed one of those is reporting the problem you are >> talking about, would be helpful. If none of those tickets reflect the >> problem you are talking about, then a complete description of the problem >> would be helpful; as it is I have no idea what problem you are referring >> to. >> >> >>> >> >> On a side note, so
Re: Let's Talk About ``to_field``
As usual, my apologies for lacking context :) The problem happens when you try to query with a model containing a foreign key that references a to_field. It fails to pass the proper attribute. # Exampe 1, a simple get lookup class ExampleBrokenModel(models.Model): user = models.ForeignKey(User, to_field="username") user = User.objects.get(username="a_username") # The following will fail due to the fact it's going to try to do user_id= user.pk, rather # than user_id=user.username ExampleBrokenModel.objects.get(user=user) I will attach additional tests to a ticket, see also [1] I'm not aware of this ever working properly, but I'm not going to dig up tickets. You can take my word for it as we've been dealing with it at this project for a year, and had been working with Django almost 2 years before that. Either way, it's not a big deal, the fact is it doesn't work as advertised, and it is advertised. You would have to talk to the author of newforms-admin, which I believe was Brian Rosner. There is some special code for handling to_field lookups in the admin. [1] http://www.pastethat.com/LUYWh David Cramer On Wed, Sep 23, 2009 at 6:27 PM, Karen Tracey wrote: > On Wed, Sep 23, 2009 at 4:59 PM, David Cramer wrote: > >> >> I haven't brought this up for quite a while, but ``to_field`` still >> does not function properly (at least as of 1.1, I haven't noticed any >> changes addressing it though). The issue is that anywhere you do a GET >> lookup, it doesn't process the to_field properly:: >> >># TODO: waiting on to_field fix >>try: >>opt = cls.objects.get(business=business.business_id, >> key=key) >>except cls.DoesNotExist: >>opt = cls.objects.create(business=business, key=key, >> value=value) >> >> > That's supposed to illustrate the problem with to_field? It's a snippet of > code lacking any context with a cryptic TODO comment. What is the model > definition here (which would at least give a clue where to_field applies)? > How would this code be different if this "to_field fix" were applied? > > This query: > > > http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&summary=~to_field&order=priority<http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&summary=%7Eto_field&order=priority> > > shows 4 open tickets with the "to_field" in their summary. Is the problem > you are referring to one of these? > > >> This seems to have been solved in the admin branch when newforms-admin >> landed, but seems mostly ignored in the core field's. This issue has >> also existed as far as I can remember, which is quite some time :) >> >> > I've not been around as long as you so without reference to a ticket or > something unfortunately I have no idea what problem you are talking about. > I also don't know what you mean by it being fixed in admin but not in "core > fields". Admin doesn't define fields, so huh? I can't really parse what > you are trying to say here. > > > >> So, what do we need to get the ball rolling on this. I don't have much >> time myself to commit, but if it's minor things such as writing some >> tests that easily reproduce all errors we've found. We'll glady commit >> those. Other than though my day is just filled, but I'd love to see >> this get fixed. It's something that's advertised as a core feature, >> has been for 3+years, and simply doesn't work. >> >> > The tests you mention filed in a ticket, or attached to one of those > tickets from above if indeed one of those is reporting the problem you are > talking about, would be helpful. If none of those tickets reflect the > problem you are talking about, then a complete description of the problem > would be helpful; as it is I have no idea what problem you are referring > to. > > >> > > On a side note, something that I also believe is a bug, is that >> ManyToManyRel.get_related_field() returns the primary key field no >> matter what. I'm not sure if thats the intended functionality of that >> or not, but to me it sounds like this should refer to the field in the >> through model (even if that model is virtual) that it's pointing to. >> > > A filed ticket describing the problem and how to recreate it would have a > better chance of getting remembered and fixed than a side note in an email. > > Karen > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Session/cookie based messages (#4604)
I'm a bit late in here, and it seems we reinvented a wheel as well (even tho we did this about a year ago), but recently just OS'd our simply notices system [1]. I'm also +1 for including something like this in trunk rather than using the current user messages. I had a brief look at django_notify and its a similar approach, but it seems much more configurable than what we wrote. [1] http://github.com/dcramer/django-notices On Sep 22, 9:49 pm, Tobias McNulty wrote: > On Tue, Sep 22, 2009 at 9:51 PM, Russell Keith-Magee > > wrote: > > In reality, I get a ping time closer to 300 ms. And that's to a > > high-end data center under ideal conditions - it can be much larger if > > I'm dealing with low end providers. > > What?? 200 ms is the average quoted by Mr. Sproutcore himself! > > No, in all seriousness, my apologies for making such a broad generalization > about packet latency. I could/should have said 500 ms or even a second; the > argument and corresponding solution, if needed, remain the same. > > Anyways..we digress. I am marking this a non-issue in my own head - please > pipe up if there's a case to be made. > > Tobias --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Let's Talk About ``to_field``
I haven't brought this up for quite a while, but ``to_field`` still does not function properly (at least as of 1.1, I haven't noticed any changes addressing it though). The issue is that anywhere you do a GET lookup, it doesn't process the to_field properly:: # TODO: waiting on to_field fix try: opt = cls.objects.get(business=business.business_id, key=key) except cls.DoesNotExist: opt = cls.objects.create(business=business, key=key, value=value) This seems to have been solved in the admin branch when newforms-admin landed, but seems mostly ignored in the core field's. This issue has also existed as far as I can remember, which is quite some time :) So, what do we need to get the ball rolling on this. I don't have much time myself to commit, but if it's minor things such as writing some tests that easily reproduce all errors we've found. We'll glady commit those. Other than though my day is just filled, but I'd love to see this get fixed. It's something that's advertised as a core feature, has been for 3+years, and simply doesn't work. On a side note, something that I also believe is a bug, is that ManyToManyRel.get_related_field() returns the primary key field no matter what. I'm not sure if thats the intended functionality of that or not, but to me it sounds like this should refer to the field in the through model (even if that model is virtual) that it's pointing to. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: 1.2 Proposal: django debug toolbar in contrib
Oh, and thats most likely my branch you're referring to Martin. I implemented a lot of the panels, and went so far as adding crazy monkey patching in some of them to catch some information I wanted. http://github.com/dcramer/django-debug-toolbar/tree/master On Aug 11, 11:38 am, Martin Maney wrote: > On Tue, Aug 11, 2009 at 09:50:35AM -0500, Alex Gaynor wrote: > > Right now django-debug-toolbar has a pretty stable panel interface and > > I actually can't recall it changing since release. A possibly more > > interesting issue is that some of the debug information it get's is > > somewhat of a hack, that is template data is gotten from monkey > > patching Template.render, the SQL data is also a monkeypatch. > > Caching stats were another area where monkeypatching seemed to be the > only alternative to an altered usage in one's source code. Back when I > spent some time using and extending parts of ddt the situation was as > described here: > > http://wiki.github.com/mmaney/django-debug-toolbar/cache-panel-workar... > > That predates the discussion I had with Rob Hudson about another fellow's > branch (can't recall whose offhand) which used a monkeypatch to add the > necessary intercept to the cache object. At one point I had hoped to > bend Jacob's ear at PyCon about this, but I got to be too busy with > other work and, in fact, never had time to drop in. Plus I had read > about how everyone was focusing on getting 1.1 out Real Soon Now and > figured it wasn't the best time for it. Months passed... --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: 1.2 Proposal: django debug toolbar in contrib
I think it's mostly only my fork (and it's forks) that vary much from trunk. I never got around to changing the framework to come in line with the master branch I think because it didn't support everything that was needed. Not sure if it does now however. On Aug 12, 4:00 pm, Rob Hudson wrote: > I'm not sure if this is the place but here are some other issues or > questions I have if this were to happen... > > * The jQuery question is a big one. I've taken strides to make the > debug toolbar interoperate with other popular JS frameworks (mootools > and prototype, namely). But if jQuery were decided against, would it > make sense to re-use some of the Javascript libraries that Django has > developed for its admin? Would it use the same admin media handler > trick? > > * Which set of plugins are included, and of those that have settings, > what are their defaults. Some defaults currently optimize to general > case and provide more debugging information, but on larger sites > performance may suffer severely. It may make sense to take a > philosophy of best default performance with opt-in added debugging. > > * Should there be a place for plugins (panels) that don't make the > cut? Panels consist of some Python and some HTML, so it's not a > single file. They *could* be changed to work similar to form.widgets > with a render method and HTML backed in... but I'm not a huge fan of > that. But it would make it a bit easier for posting on a site like > Django Snippets. > > * Should the UI match that of the Django admin (overall theme, colors, > etc)? > > * If the debug toolbar is incorporated, should we consider different > integration options? e.g. Currently the debug toolbar injects URLs > into the project in a rather ugly way -- could this be done better if > it were part of Django? > > Thanks, > Rob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Why does get_profile exist?
I was never a fan of the profile model as it stands. It's not very practical in every situation I've ever been in. Being that 1.0 was supposed to be backwards compatible, and this is a public API I think it needs to stay. I'd love to see a way to swap out the User model in the future though, rather than attach an additional db call for profiles. However, the one-to-one approach is a lot nicer than get_profile(). On Apr 13, 11:27 pm, Glenn Maynard wrote: > On Tue, Apr 14, 2009 at 12:00 AM, James Bennett wrote: > > Well, first of all user profiles aren't a "narrowly useful special > > case" -- they're an extremely common feature needed on lots of > > real-world sites. So having some sort of standard API for that is a > > good thing. > > Obviously, I didn't say user profiles were narrow; I said the benefit > of having this special case is limited. > > > Second, I'm not sure why you're devoting so much energy to what's > > I'm not. I keep saying it's not a big deal, doesn't really need to be > changed, and that I just wanted to know how it got that way; you keep > writing essays in response. I'm going to elide the rest so I don't > get another essay dropped on my head. > > > Personally I'm still a bit wary of OneToOneField. Not for any reason > > related to the code or technical issues, but simply because of what it > > implies -- OneToOneField is most useful for handling the link between > > two classes in a multi-table inheritance situation (which is how > > Django makes use of it internally), and any time I see a OneToOneField > > my instinctive reaction is "oh, this is meant to behave like a > > subclass of that other model". > > I find the "zero or one to one" case much more commonly useful, and > OneToOneField is much more natural for that (because ForeignKey's > reverse gives a set); but that's the subject of another thread (and, > of course, that topic is what led me to wonder about this one). > > -- > Glenn Maynard --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: django.contrib.sitemaps Additions and Tweaks
Actually before I get a "wtf" response, I should note that the two generators do not quite match. One is rendering a template, the other is simply yielding XML. The latter approach is what I prefer, but I was rushing to finish the index generator :) On Jan 14, 11:49 am, David Cramer wrote: > While the Sitemaps framework in Django is fairly complete, there's a > few things it's lacking, namely the ability to specify attributes for > a Sitemap index file. We ended up rewriting our sitemaps, but I wanted > to throw our code up here for an example of what we came up. Keep in > mind that we are writing static sitemaps, and currently have not > implemented a way to automatically update them, beyond regeneration. > > http://www.pastethat.com/dc01x/syntax/python/python > > The first change which we made was to convert the entire process of > creating a sitemap into a generator. While I'm not sure if Django can > actually use generators in the response, it accepts them and that > optimization can be left up to the core this way. When we are > generating static site maps, this does however help with performance. > If the core can somehow use the generators as an optimization, this > would be an amazing change to have, as that was the reason we had to > use static files. > > Next we changed sitemaps always force instantiation in urls.py > (instead of just passing the class reference). For our use cases we > don't need to verify if it's callable, and it makes sense in our > environment to keep the instance in memory since it's small. This > probably isn't something that would fit well into trunk, especially > since its backwards incompatible. > > Lastly, we added index attributes, which I believe are the most > important. We have hundreds of sitemaps so we were using the sitemap > index feature. This was great, until we needed the ability to specify > a changefreq on some of the indexes (e.g. indexes that are updated > each day). We also wanted to specify the lastmod date to help Google. > In the end, we simply modified the Sitemap class to include > idx_ for the index attributes. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
[Proposal] django.contrib.sitemaps Additions and Tweaks
While the Sitemaps framework in Django is fairly complete, there's a few things it's lacking, namely the ability to specify attributes for a Sitemap index file. We ended up rewriting our sitemaps, but I wanted to throw our code up here for an example of what we came up. Keep in mind that we are writing static sitemaps, and currently have not implemented a way to automatically update them, beyond regeneration. http://www.pastethat.com/dc01x/syntax/python/python The first change which we made was to convert the entire process of creating a sitemap into a generator. While I'm not sure if Django can actually use generators in the response, it accepts them and that optimization can be left up to the core this way. When we are generating static site maps, this does however help with performance. If the core can somehow use the generators as an optimization, this would be an amazing change to have, as that was the reason we had to use static files. Next we changed sitemaps always force instantiation in urls.py (instead of just passing the class reference). For our use cases we don't need to verify if it's callable, and it makes sense in our environment to keep the instance in memory since it's small. This probably isn't something that would fit well into trunk, especially since its backwards incompatible. Lastly, we added index attributes, which I believe are the most important. We have hundreds of sitemaps so we were using the sitemap index feature. This was great, until we needed the ability to specify a changefreq on some of the indexes (e.g. indexes that are updated each day). We also wanted to specify the lastmod date to help Google. In the end, we simply modified the Sitemap class to include idx_ for the index attributes. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: autodiscover for contrib.sitemaps
It would be kind of neat to have some generic autodiscover functionality for this kind of stuff. Not only could you reuse it in things like sitemaps, templatetags, and admin, but it could be used by other developers for plugable systems of their own. On Dec 19, 4:01 am, andybak wrote: > I have recently been working on an app that was similar enough in > functionality to contrib.sitemaps for me to use that code as a base. > > It struck me that the way you register apps with sitemaps was a little > clunky and could use a bit more DRY . > > Is there any reason it couldn't adopt the same approach as > admin.autodiscover? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Exception emails in django.core.handlers.base
We simply send them to localhost. Learned the lesson long ago to not spam a normal email account with it :) On Dec 16, 4:15 pm, "Jeremy Dunck" wrote: > On Tue, Dec 16, 2008 at 3:44 PM, Jacob Kaplan-Moss > > > > wrote: > > > On Tue, Dec 16, 2008 at 11:56 AM, Jeremy Dunck wrote: > >> Since sending email can block for an arbitrarily long time, I'd like > >> to make it so that when an exception occurs in > >> django.core.handlers.base, it calls our mail_admins rather than > >> django's stock mail_admins. > > >> With an eye towards backwards-compatibility, I can't see an easy way > >> to do this other than to introduce a new setting. > > >> settings.EMAIL_MODULE = 'django.core.mail' > > > Seems it'd be much easier to just subclass the appropriate handler > > (ModPythonHandler or WSGIHandler) and use that instead of Django's > > default handler. The core handler's written in a pretty modular way by > > design; this kind of subclassing ought to be pretty easy. > > In this case, you're suggesting I subclass *Handler and override > get_response in order to change the except clauses which call > mail_admins? > > I'm just trying to be sure I understand what you're saying. > > > Plus I'm not sure how a EMAIL_MODULE setting would work -- would it > > really sneak in and force anyone who imports from django.core.mail to > > get EMAIL_MODULE instead? > > Nothing that smart-- just that Django's core code would use whatever > module was named by settings.EMAIL_MODULE rather than importing > django.core.mail directly. > > If people are happily using django.core.mail, more power to them-- I > just wish django didn't assume that for me. :-) > > Put this another way-- how do other people manage high-performance > exception mailing? Maybe I'm Doing It Wrong. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Custom Fields API Cleanup
Could we just use a lazy proxy for the value value? When it's executed, it could then just call the get_value_from_db method (or whatever). On Tue, Dec 16, 2008 at 12:37 PM, Marty Alchin wrote: > > On Tue, Dec 16, 2008 at 1:03 PM, Jacob Kaplan-Moss > wrote: > > This seems like a good idea to me, and an easy one to implement; I'm > > all for it. Are there any backwards-compatibility concerns here? It > > doesn't look like it -- we're just talking about adding a new hook to > > SubFieldBase, right? > > My only concern with this proposal so far is that I fear it'll fall > short of what many people will really be looking to do here. Every > time I've done this dance, I ended up something a bit more > complicated, because I also needed to delay the processing of the db > value until it's actually used. Basically, the same idea behind > implicit ForeignKey lookups and lazy geometries in GIS. > > While I do realize that there are use cases where you'd want the > processing to take place up front, when first loaded from the > database, it seems like a slippery slope, since there are also many > cases where you'd want to delay that processing. Since David's > proposal doesn't address delayed processing, we'd have to make sure to > manage expectations so people don't cry foul when it still doesn't do > what they're looking for. > > I've been all for expanding how data conversion in fields is handled, > and I think SubfieldBase is a great first step for common cases. I > just wonder how many of the less-common cases we plan to address > before we start getting into support code that's bigger and harder to > maintain than custom solutions. I'm fine with the line being pushed > back somewhere so we can get more support in core, but I do want to > make sure a line gets drawn somewhere. > > Given that I've had a few encounters with this type of thing before, > I'll gladly help out where I can, but I'd also want to make sure > Malcolm weighs in, since I think he was intentionally conservative in > determining just how much SubfieldBase was responsible for. > > -Gul > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Custom Fields API Cleanup
I should also note, that these changes, could possibly be carried over into Model Field's. As IIRC they were suffering from some of the same problems. On Tue, Dec 16, 2008 at 12:03 PM, Jacob Kaplan-Moss < jacob.kaplanm...@gmail.com> wrote: > > On Tue, Dec 16, 2008 at 9:06 AM, David Cramer wrote: > > The first of which, is the pre_save method. Originally we had been > > using get_db_pre_value (which also is passed on to the save method), > > and this seems to make a lot more sense than pre_save's > > implementation. I'm not 100% sure which one is the "preferred" method, > > but since examples are running around on the internet using both. It'd > > make sense to iron out some of the details: > > This is probably just a matter of adding some appropriate verbiage to > the custom fields documentation indicating which is "preferred" -- or > perhaps explaining why you'd choose one or the other -- but that bit > of docs would go a long way, I think. I can't say I know off the top > of my head, frankly. > > > Now if we were to add some kind of API which makes it easier to handle > > the "post_init" as seen above: > > > > class JSONField(models.TextField): > >__metaclass__ = models.SubfieldBase > > > >def get_value_from_db(self, value): > >if not value: return None > >return simplejson.loads(value) > > > >def get_db_prep_value(self, value): > >if value is None: return > >return simplejson.dumps(value) > > > > Overall, the changes would be to simply provide more explicit hooks in > > the API. > > This seems like a good idea to me, and an easy one to implement; I'm > all for it. Are there any backwards-compatibility concerns here? It > doesn't look like it -- we're just talking about adding a new hook to > SubFieldBase, right? > > Jacob > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Custom Fields API Cleanup
Recently we've been doing a lot of work with custom fields, and have had a few inconveniences come up. Some minor, some not so much. I'd like to propose a few changes to the API to make it easier to write custom fields, especially those which use custom data structures. The first of which, is the pre_save method. Originally we had been using get_db_pre_value (which also is passed on to the save method), and this seems to make a lot more sense than pre_save's implementation. I'm not 100% sure which one is the "preferred" method, but since examples are running around on the internet using both. It'd make sense to iron out some of the details: class ModifiedDateTimeField(DateTimeField): def pre_save(self, model_instance, add): value = datetime.datetime.now() setattr(model_instance, self.attname, value) return value This is similar to an example found on the web, which gives you a DateTimeField which automatically updates when the model is saved. The only real issue I see with the pre_save method here, is that you're still required to do setattr() in order to change the value, even if you return a new value. Unless there is a specific reason for this, it would make sense to have it automatically set the attribute on pre_save. It seems, based on some examples, that this may be used strictly to modify what is being saved to the database, but the name doesn't really imply that. The more pressing problems I've had, are with custom datastructures. We use a few different field types, such as a ListField, SerializedField, and a JSONField. Each of these uses a custom data type. The issue with these, that, while it's easy to serialize the data into the database (using get_db_pre_value), it's not as easy to manage the value when it's taken from the database, or set directly. I'd propose two new hooks, which work like to_python. One would be called to set the data when it's pulled out of the database, and the other would be called when it's set normally. Possibly even just leaving to_python as this, and add the new database hook. Here is a rough example, of the current implementation to make this work: (source: http://www.djangosnippets.org/snippets/377/) class JSONField(models.TextField): def db_type(self): return 'text' def pre_save(self, model_instance, add): value = getattr(model_instance, self.attname, None) return dumps(value) def contribute_to_class(self, cls, name): super(JSONField, self).contribute_to_class(cls, name) dispatcher.connect(self.post_init, signal=signals.post_init, sender=cls) def get_json(model_instance): return dumps(getattr(model_instance, self.attname, None)) setattr(cls, 'get_%s_json' % self.name, get_json) def set_json(model_instance, json): return setattr(model_instance, self.attname, loads(json)) setattr(cls, 'set_%s_json' % self.name, set_json) def post_init(self, instance=None): value = self.value_from_object(instance) if (value): setattr(instance, self.attname, loads(value)) else: setattr(instance, self.attname, None) This handles setting it from the database just fine, and you don't need to worry about to_python (as we're keeping the original data structure). The problem with this example, is it requires using a signal, and IMHO is a lot more work that it could be. Now if we were to add some kind of API which makes it easier to handle the "post_init" as seen above: class JSONField(models.TextField): __metaclass__ = models.SubfieldBase def get_value_from_db(self, value): if not value: return None return simplejson.loads(value) def get_db_prep_value(self, value): if value is None: return return simplejson.dumps(value) Overall, the changes would be to simply provide more explicit hooks in the API. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Additional Fields in Core
Ya core/contrib (same thing to me). My biggest issue with custom fields at the moment is the fact that the hooks aren't really the best. JSONField/etc have to use ugly signals to allow any data-type to be used. On Dec 15, 4:12 pm, "Mike Axiak" wrote: > I'd much rather see a contrib app for these. E.g.:: > > from django.contrib.datafields import fields as datafields > > class MyModel(models.Model): > my_yaml = datafields.YAMLField() > ... > > and any refactoring that's necessary to facilitate this cleaning should be > dealt with. (And I think if they subclass TextField they are reasonably easy > to do without too much backend awareness.) > > My $0.02, > Mike > > On Mon, Dec 15, 2008 at 5:07 PM, David Cramer wrote: > > > I'd like to propose the addition of some useful (and common) fields in > > the django core. Things like a JSONFied, YAMLField, HTMLField?, > > UUIDField, etc. > > > A lot of these seem like common things people use, and while many > > examples are out there, it'd be nice to simply include these in core > > so there's a defacto standard. > > > Maybe at the same time, clean up the Field API if possible :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Additional Fields in Core
I'd like to propose the addition of some useful (and common) fields in the django core. Things like a JSONFied, YAMLField, HTMLField?, UUIDField, etc. A lot of these seem like common things people use, and while many examples are out there, it'd be nice to simply include these in core so there's a defacto standard. Maybe at the same time, clean up the Field API if possible :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: How do you handle cascading deletes in your production apps?
To be perfectly honest, I'm +1 on a solution. I didn't realize this was happening either. Time to monkey-patch QuerySet's delete() method :) (I'm also a bit curious as to how its handling cascades in MySQL when it's not looping through each object) On Dec 10, 12:43 pm, "Karen Tracey" <[EMAIL PROTECTED]> wrote: > On Wed, Dec 10, 2008 at 12:59 PM, AcidTonic <[EMAIL PROTECTED]> wrote: > > > There is no anger here sir. I'm a rather sarcastic individual and I > > guess there was a good dose there. > > > I just cant for the life of me understand why this excellent framework > > completely missed the point there. > > > Is there a workaround that fixes the problem vs working around it? As > > in a django patch/hack/fix as opposed to my code needing a change? > > I've found that searching here: > > http://code.djangoproject.com/search > > for likely keywords (in this case I'd try "cascade") and reading the cited > tickets, etc. is a good way to get a feel for the history of features that > are perhaps non-obvious/surprising to me. (Also doing similar searches on > the mailing lists can be useful.) I'll get a sense of whether it's just me > or if there's been a lot of back-and-forth discussion and what influenced > the way things are now. In this case as Joey already mentioned you'll find > a ticket with a patch (that may not be current) that may be proposing the > kind of thing you are looking for in the framework. You might want to > evaluate that approach and see if it works for you, if not propose an > alternative, etc. > > Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Caching over 1MB
Oh no, now I'm going to get more emails saying the code needs updated :) There is a concept in the orm cache that splits it though, so you can probably use that idea. On Dec 7, 10:04 am, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > On Sun, Dec 7, 2008 at 8:55 AM, Calvin Spealman <[EMAIL PROTECTED]> wrote: > > > Using the caching api backed by memcache, i started hitting the 1MB > > object size limit of memcache and I was thinking of a way around it by > > storing a list of keys to segments of the full object, each up to 1MB > > each. I figure this will raise the limit to 15GB and that had better > > be large enough for anyone. > > > But is it worth it? Should I work within the limits and be glad for > > them keeping me on my toes? > > This may interest you:http://code.google.com/p/django-orm-cache/ > > Also, you can recompile memcache to have an arbitrary value size limit. > > But in general, yes, finer-level cache objects is a good way to go. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: BitmaskField
Well you could do a tuple, but really if the programmer doesn't know what a Bitmask is, they should not be using it. They're by far not a simple technique, and should only be used by those who understand when and why they benefit. On Dec 6, 3:53 pm, "Craig Kimerer" <[EMAIL PROTECTED]> wrote: > Andrew: Thanks, that looks awesome. > > The whole BitMaskField(choices=LIST) idea scares me. You must then force > extra knowledge on the user that ordering is important. If programmer Y > decides the list of choices looks better in alphabetical order (or decides > to add a choice in the middle of the list), any data that is already in the > table becomes incorrect. > > I guess you have the same problem with people deciding to re-number the > choices, and you can just as easily leave a comment above it that ordering > is important and all new choices should be added to the end. Another option > is to store the choices in something like the django_content_type field and > verify on syncdb (or some other place, not sure where it would go) that any > old field types were not assigned to a new mask. > > Craig > > On Sat, Dec 6, 2008 at 1:12 PM, David Cramer <[EMAIL PROTECTED]> wrote: > > > Awesome to see some people working on this. I had tried pre-queryset > > refactor and It was just not doable witht he fields API. Can't wait to > > see the final result of this :) > > > I'm also agreeing with the API of field = BitMaskField(choices=LIST) > > > On Dec 6, 10:37 am, Carl Meyer <[EMAIL PROTECTED]> wrote: > > > @Andrew: Thanks! That's precisely the missing piece from my code; if > > > I get some time to put it all together, I think it'll be a full > > > solution. My approach uses sets of arbitrary flag values rather than > > > creating constants for flags, and it's implemented as a normal model > > > field, which seems a little cleaner than introducing a new syntax. > > > > @Craig: I never thought of just monkeypatching the necessary > > > dictionaries to add new lookup types. You'd need to do it for all the > > > DB backends, of course. And as with monkeypatching in general, it's > > > not a real sustainable practice to go messing with global reference > > > dicts; if it became common practice to do that for new field types > > > you'd start getting name collisions pretty quickly. I think I'll > > > probably continue to prefer the custom Q object. > > > > Carl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: SessionWizard
Would have to look at how FormPreview's work, but I agree. So far what I've done with the wizard is remove all of the data from the session as much as possible. It calculates cleaned_data at the very end and only stores the POST values (still need to solve a potential exploit). One thing I didn't like was storing the base_forms in the session (in order to dynamically change the forms based on what has happened. The way this is done it would severely bloat the Form version, or make it not possible. Anyone have suggestions on a way to handle this? We could calculate the list of forms based on the data stored each time, but not really sure. On Dec 4, 5:23 am, "Hanne Moa" <[EMAIL PROTECTED]> wrote: > On Thu, Dec 4, 2008 at 03:34, David Cramer <[EMAIL PROTECTED]> wrote: > > When the done() method is called, the session is also cleared unless > > done(). Maybe this should only happen if say a ValidationError isn't > > raised? Maybe a setting that makes this not happen? In all of my > > situations it makes sense to erase it when it's complete, and not have > > to call methods manual. Let's decide which is more common. > > How about making it easier to have a preview also? As per today, > FormWizard and FormPreview aren't really combinable. Should probably > be it's own ticket... > > HM --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: BitmaskField
Awesome to see some people working on this. I had tried pre-queryset refactor and It was just not doable witht he fields API. Can't wait to see the final result of this :) I'm also agreeing with the API of field = BitMaskField(choices=LIST) On Dec 6, 10:37 am, Carl Meyer <[EMAIL PROTECTED]> wrote: > @Andrew: Thanks! That's precisely the missing piece from my code; if > I get some time to put it all together, I think it'll be a full > solution. My approach uses sets of arbitrary flag values rather than > creating constants for flags, and it's implemented as a normal model > field, which seems a little cleaner than introducing a new syntax. > > @Craig: I never thought of just monkeypatching the necessary > dictionaries to add new lookup types. You'd need to do it for all the > DB backends, of course. And as with monkeypatching in general, it's > not a real sustainable practice to go messing with global reference > dicts; if it became common practice to do that for new field types > you'd start getting name collisions pretty quickly. I think I'll > probably continue to prefer the custom Q object. > > Carl --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: SessionWizard
So far I've refactored a bunch of the methods to store less in the session, and generate more on demand (otherwise you could change the method and then session data represents inaccurate information). I've completedly redesigned the pages storage so it's just storing valid and verified in the session. I'm a little curious as to what valid is (vs verified), but I'll leave it for now. I've fixed an issue with URL base, it seemed to only allow URLs which didn't end with a / (mine do). (r'^verify-number/(:?(?P\w*)/)?$', 'verification.index'), This is my sample URL. I've changed the done() method to pass form_classes just as it would originally. I've also changed _validate_all_forms to support this change in the new _handle_post_request (renamed from POST) method. GET is renamed to _handle_get_request. get_URL_base is renamed to get_url_base. When the done() method is called, the session is also cleared unless done(). Maybe this should only happen if say a ValidationError isn't raised? Maybe a setting that makes this not happen? In all of my situations it makes sense to erase it when it's complete, and not have to call methods manual. Let's decide which is more common. On Dec 3, 7:50 pm, David Cramer <[EMAIL PROTECTED]> wrote: > So I needed the ability to specify initial data for a wizard, and I > also liked the idea of storing it in the session (vs the POST). I was > pointed to the SessionWizard patch, and I believe this was a 1.1 > possibility as well, so I figured I'd bring up the numerous issues. > > First and foremost, the latest patch doesn't even work. There's > several naming conflicts, and mismatches in the docs. There's a few > things I like about this, that I think would work well in the base > wizard class (which needs to be created): > > - A few more hooks such as get_page_title > - The possibility to interact with the GET request to get form data (I > want to sometimes skip a step if they come from a location which > provides that information). > > And a few of the obvious issues: > > - The API is *completely* different. > - Much of the code is specifically written for sessions (which is good > and bad). > > I've got it working locally myself, but I've already begun making > tweaks (to make it work in our environment, and fix the bugs). I'm > going to refactor a BaseWizard class, and try to keep all the original > FormWizard methods, such as render_template, done, etc.. > > Most of the original base methods are still possible without tweaking > the design, but one of the very obvious differences is the done > method. This may be up for debate, but I believe the done method > should be what it says, done. This means that the session data should > automatically wipe once the done method has been called. There is also > the issue that the forms are not passed via the done method, and this > needs changed so that it is the same API. > > The API (even private) also needs cleaned IMO to be a bit closer to > PEP 8. > > Anyways, I'm going to work up some changes, and I'll submit another > patch. If anyone has any recommendations on API for this please let me > know as I can commit the time nescesary (since we need this) to finish > the code. > > Also, is this feature still slated for 1.1? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
SessionWizard
So I needed the ability to specify initial data for a wizard, and I also liked the idea of storing it in the session (vs the POST). I was pointed to the SessionWizard patch, and I believe this was a 1.1 possibility as well, so I figured I'd bring up the numerous issues. First and foremost, the latest patch doesn't even work. There's several naming conflicts, and mismatches in the docs. There's a few things I like about this, that I think would work well in the base wizard class (which needs to be created): - A few more hooks such as get_page_title - The possibility to interact with the GET request to get form data (I want to sometimes skip a step if they come from a location which provides that information). And a few of the obvious issues: - The API is *completely* different. - Much of the code is specifically written for sessions (which is good and bad). I've got it working locally myself, but I've already begun making tweaks (to make it work in our environment, and fix the bugs). I'm going to refactor a BaseWizard class, and try to keep all the original FormWizard methods, such as render_template, done, etc.. Most of the original base methods are still possible without tweaking the design, but one of the very obvious differences is the done method. This may be up for debate, but I believe the done method should be what it says, done. This means that the session data should automatically wipe once the done method has been called. There is also the issue that the forms are not passed via the done method, and this needs changed so that it is the same API. The API (even private) also needs cleaned IMO to be a bit closer to PEP 8. Anyways, I'm going to work up some changes, and I'll submit another patch. If anyone has any recommendations on API for this please let me know as I can commit the time nescesary (since we need this) to finish the code. Also, is this feature still slated for 1.1? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Make django debug page simple when develop ajax page(with firebug)
I'll agree 100% that a request.is_ajax() hook to output some simplified data would be awesome. On Dec 2, 10:42 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Perhaps a better solution would be to vary based on request.is_ajax(). > > Alex > > On Dec 2, 11:37 pm, kernel1983 <[EMAIL PROTECTED]> wrote: > > > HTML debug page is good when you develop normal web page > > > But when you debug ajax based web page with firebug, the HTML debug > > page is no good for you. > > > I did some hack in django/views/debug.py > > > I remove TECHNICAL_404_TEMPLATE TECHNICAL_500_TEMPLATE html tags and > > make it simple. > > > I think we can have a configure key in settings in the future. The > > developers can choose which style of debug page to use, by which type > > application they're developing. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: deprecate and remove django.utils.simplejson
I have nothing against removing it from the built-in libs, but as long as we look for the system's libraries first what's the big downside to keeping it? On Dec 1, 5:02 am, "Russell Keith-Magee" <[EMAIL PROTECTED]> wrote: > On Mon, Dec 1, 2008 at 4:21 PM, James Bennett <[EMAIL PROTECTED]> wrote: > > > On Mon, Dec 1, 2008 at 12:45 AM, Russell Keith-Magee > > <[EMAIL PROTECTED]> wrote: > > I guess the thing that's bugging me is that this mostly seems to come > > down to historical inertia; we already have simplejson in Django, and > > so the perception is that we should continue to bundle it. I think > > eventually we're going to have to get rid of it since sooner or later > > we'll be targeting only Python versions which have a json module built > > in, so we might as well start dealing with it. > > Granted - the Python 3 migration plan will eventually force the issue. > I just don't see any urgency to address it before then. If we link the > two schedules, we can end up in a situation where SimpleJSON is > guaranteed to be available while deprecating our own maintained > version. > > >> On the other hand: at present, the only prerequisite for running the > >> system test suite is a database backend, and not even that if you're > >> running Python 2.5 and SQLite. If we stopped packaging SimpleJSON with > >> Django, a great chunk of the system test suite would no longer work > >> out of the box. On top of that, no matter obvious we make the error > >> message, we're going to get "Why doesn't my fixture load" questions on > >> Django-users. > > > Several large and popular third-party Django apps already make use of > > non-JSON fixture formats; for example, Satchmo cannot load its > > fixtures or run its unit tests without PyYAML (and in my experience, > > YAML is preferred over JSON as a fixture format when it can be used), > > and yet they don't seem to be crushed under the weight of "I can't > > load the fixtures" problems. > > Django community != Satchmo community. I would be very surprised if > the Python competence of the average Satchmo newbie wasn't > significantly higher than that of the average Django newbie. By the > time a new developer has worked out what Satchmo is, and how it fits > into Django, one would hope that they have also worked out that this > is all just Python code, and the error messages really are telling you > what is wrong. > > > Similarly, ImageField has always required PIL (which is much trickier > > to install), and yet we don't see corresponding large numbers of > > questions on this list from people who can't manage to get it working. > > Firstly, there aren't that many tests that depend on PIL. By contrast, > every test with a fixture depends on SimpleJSON. > > Secondly, in most (all?) of the tests where PIL is required, the test > downgrades the ImageField to a FileField if PIL isn't available. This > does lead to slightly less complete test coverage, but not failed > tests. A similar downgrade would not be possible if SimpleJSON were > absent without providing fixtures in alternate formats or disabling > half the test suite. > > > So I have a hard time believing that this would pose such a large > > hurdle to use of Django. > > I wouldn't have thought an error message that reads "Error: Settings > cannot be imported, because environment variable > DJANGO_SETTINGS_MODULE is undefined" would lead to so many questions > on the mailing list, and yet > > >> IMHO, the suggestions hovering around #9266 and the related mailing > >> list threads - that users should be able to override the bundled > >> version of SimpleJSON - have merit, and it looks like Malcolm has a > >> handle on how to make this approach happen. However, completely > >> removing SimpleJSON seems like asking for a world of pain, with no > >> real gain. > > > To me, the gain is no longer bundling and maintaining something that > > we don't *have* to bundle and maintain. Advertising simplejson (or > > Python 2.6+) as a dependency for using JSON serialization doesn't, to > > me, seem to be too onerous a thing to do, and it gets something out of > > Django (where, arguably, it didn't belong in the first place) and out > > of our hair. > > My point (and Malcolm's) is that while this is true, the burden of > maintaining SimpleJSON simply isn't that high. In all honesty, I've > probably spent more effort writing emails on this thread that I have > dealing with SimpleJSON issues in the last 3 years. On the other hand, > I have spent a lot of time answering newbie questions by pointing them > at obvious error messages. > > Yours, > Russ Magee %-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-devel
Re: Dropping Python 2.3 compatibility for Django 1.1
+1 Update your damn distros. Generators are important :) On Nov 26, 10:33 am, "Gary Wilson Jr." <[EMAIL PROTECTED]> wrote: > On Wed, Nov 26, 2008 at 7:23 AM, varikin <[EMAIL PROTECTED]> wrote: > > On Nov 25, 7:16 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> > > wrote: > >> However, even saying Django 1.1 is the last 2.3-compatible version and > >> we drop it afterwards gives us a reasonable 3.0 support timeline, since > >> our timeframe doesn't really encourage any official 3.0 support for 1.1. > > > I am +1 to saying 1.1 is the last release for 2.3 (or just deprecated > > and dropped sometime in the future). Pulling support for something is > > large step and was never discussed for 1.1 openly till now. Anyone who > > has read the roadmap but not following any more than that could be in > > for a nasty surprise. > > Actually, dropping 2.3 support was openly discussed for 1.0 at PyCon > 2008. In a room with at least 60 developers, I was the only one that > raised my hand when Jacob asked about people using Python 2.3 (I had > RHEL4 boxes in production). Also at PyCon, the core developers later > decided to keep 2.3 support for 1.0 and drop it shortly afterwards > (Jacob, looks like we still need to post those meeting > minutes:http://www.djangoproject.com/foundation/records/). > > Digging through my notes here, it seems that a big reason for keeping > Python 2.3 support for 1.0 was for the benefit of the > Jython/IronPython/PyPy folks. I'm not sure how these areas have come > along since then. jython.org seems down at the moment, but from the > IronPython page, I gather they are at CPython 2.4 and 2.5 > compatibility levels [1] with their 1.x and 2.x releases, > respectively. If Python 2.3 support still helps these folks then then > I would be in favor of keeping 2.3 support around for 1.1. > > Otherwise, I'm all for dropping 2.3 support, as maintaining 2.3 > support is not fun at all. Just dig through changesets and note all > of the 2.3 bugs that were introduced and fixed over the last major > development cycle, for example. Python 2.3 unicode bugs have been the > most annoying, but there have been a few rsplit, list generator, and > other syntax bugs as well. Testing is made easier, too, since it > means one less version of Python to test against. > > As for the roadmap, I think that is the point of this discussion. We > are finalizing the features for 1.1, and if Python 2.3 support is to > be dropped, then this fact will indeed need to be noted on the roadmap > along with the other planned features. Those needing to stay on > Python 2.3 could just keep to 1.0.x, not unreasonable if you ask me. > Also, 1.1 is still four months away and 1.0.x will be receiving bug > fixes until then, so Python 2.3 users wouldn't be completely left in > the dark. > > Gary > > [1]http://www.codeplex.com/IronPython/Wiki/View.aspx?title=Differences --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: defer().
Well I guess I'm not too familiar with how the Django process goes from proposal to actual feature, but I was looking for something a bit more concrete. Something that says "we're going to use this as an API, here is how it will work, now let's get to coding". On Nov 22, 4:13 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > On Sat, Nov 22, 2008 at 3:50 PM, David Cramer <[EMAIL PROTECTED]> wrote: > > Ok with all of that being said. I've checked the ticket (once again), > > and there doesn't seem to be any API proposed for fields as include- > > only. > > From the ticket description: > > > We should also provide the inverse of hide() -- perhaps called expose()? -- > > which would > > take a list of field names to *include* rather than *exclude*. This would > > be an opt-in > > instead of an opt-out. > > Additionally, ``fields(*fieldnames)`` and ``defer(*fieldnames)`` have > been mentioned at least a half dozen times in the ticket and the > django-developer threads -- threads you've participated in, FWIW. > That's as much of a "spec" as we need, don't you think? > > David, I know you're not really this dense. Is there something else > going on here that makes you keep harping on this issue? > > Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: defer().
Ok with all of that being said. I've checked the ticket (once again), and there doesn't seem to be any API proposed for fields as include- only. Is there another ticket other than #5420? On Nov 21, 7:15 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Fri, 2008-11-21 at 12:53 -0800, David Cramer wrote: > > I agree. The internals are shared among both, that's why I'd much > > rather see a good API around inclusion/exlusion. > > And reading the ticket would have revealed that it discusses both > defer() and an opposite. > > > I also agree that > > excluding text fields is a use-case, but this could also just be > > considered avoidable database design. At the same time, this would > > make much more sense at a model-level if you are doing this kind of > > thing, and then specifying an "include defered field" somehow in an > > API. > > As mentioned in a number of previous threads, it makes less sense, > because it's patching over a data modelling flaw which could have been > done properly in the first place. If you're trying to avoid retrieving a > field from a table *all the time*, then don't put it in the table > initially. That's just sensible data modelling. Arguments like "but > sometimes I want to retrieve it" are weak, because the amortised cost of > the join the few times you want to do so is small. All that aside, if > somebody still wants to model their data as everything in one table and > yet never retrieving certain fields, that's why we have custom managers > and the ability to override get_query_set(). > > Model-level specification specify static attributes of how the data is > stored. Equivalent to database level things. There's no overwhelming > reason not to keep consistency there. > > You have to at least stop pretending that this kind of thing is a > no-brainer just because you personally aren't interested in it. It > devalues your contributions. > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: defer().
I agree. The internals are shared among both, that's why I'd much rather see a good API around inclusion/exlusion. I also agree that excluding text fields is a use-case, but this could also just be considered avoidable database design. At the same time, this would make much more sense at a model-level if you are doing this kind of thing, and then specifying an "include defered field" somehow in an API. On Nov 21, 2:39 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Well the hard part is getting the mechanisms in there, the original > ticket(which is on the 1.1 list) requests both, but as long as we have > the internals of how that works, the rest is easy, it's just a matter > of fields = included_fields vs. fields = [x for x in > self.model._meta.fields if x not in excluded_fields]. > > On Nov 21, 3:23 pm, "Ian Kelly" <[EMAIL PROTECTED]> wrote: > > > On Fri, Nov 21, 2008 at 1:05 PM, David Cramer <[EMAIL PROTECTED]> wrote: > > > I won't use defer, and I won't recommend people use it. I think it's > > > not good practice. It's like using .select_related() implicitly. You > > > really need to explicitly pass things around in order ot keep things > > > performant. Otherwise you're going to need to update your defer() > > > statements anytime a model would change. > > > I think it depends how you're using it. If you're aggressively trying > > to retrieve only the fields you absolutely need, then an opt-in > > approach is the way to go. If you're less concerned about that, and > > you're only trying to prevent the TextFields from loading, then it > > makes more sense to specifically opt-out those fields -- especially if > > you can't predict exactly which fields will be useful and you want to > > minimize the total number of queries. > > > #4186 in particular provides a use case for defer: in Oracle, you > > currently can't use QuerySet.distinct() if your query includes a > > TextField (and even if you could, it's a bad idea in general). Defer > > would provide a convenient workaround for that. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
defer().
To avoid messing with the 1.1 Roadmap anymore. What happened with defer (). I know personally myself, and several others who expressed opinions, wanted more than just an exclude option, but an include-only option. Is this part of the "Exclude fields in a SELECT (QuerySet.defer())" or is this just ignored? It makes sense to solve both problems in one shot, instead of introducing two separate APIs to solve them. I won't use defer, and I won't recommend people use it. I think it's not good practice. It's like using .select_related() implicitly. You really need to explicitly pass things around in order ot keep things performant. Otherwise you're going to need to update your defer() statements anytime a model would change. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
You guys are really bad about keeping on topic. :) On Wed, Nov 19, 2008 at 9:32 AM, Joey Wilhelm <[EMAIL PROTECTED]> wrote: > Hanne, > > I ran into that exact problem myself and filed the following ticket related > to the issue: http://code.djangoproject.com/ticket/9519 > > I also found that you could run your own delete operation through the ORM, > however, to work around this. It's more work and certainly not in line with > DRY or any of the normal sensibilities... but it does at least work. Here's > an example of that, built off the example in the ticket: > http://dpaste.com/92116/ > > On Wed, Nov 19, 2008 at 05:07, Hanne Moa <[EMAIL PROTECTED]> wrote: >> >> Funny thing though, got a manytomany-table without a primary key: I >> can add a new row easily enough, the only problem is in delete()-ing >> specific rows, since that seems to use the primary key. So I clear() >> and re-add instead of delete()-ing, since that doesn't use the primary >> key. Fragile, but it beats doing the puppy-eyes to the DBA. >> > > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Post about django optimizations
It would be a lot more useful if you put details in to it. Right now this post looks like spam on the developers group. I'm curious as to your issues with the Paginator class. Works great for me (barring some additional context vars). On Nov 17, 11:43 pm, Dipankar Sarkar <[EMAIL PROTECTED]> wrote: > Hi guys, > > I wrote a post on django platform optimizations which i learned from > building (still learning actually) kwippy (http://kwippy.com) , i > would love to hear more tips from you guys out there :). > > http://www.desinerd.com/blog/technical/django-optimizations-within-th... > > Looking forward to *some* response. > > Dipankar > [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django documentation index redesigned
My only opinion, is beyond the tutorial, people would much rather search then dig for the link :) On Nov 18, 10:54 am, "Clint Ecker" <[EMAIL PROTECTED]> wrote: > Aw, I had actually started to like/get used to the old-new way ;) > > > > On Tue, Nov 18, 2008 at 1:44 AM, Adrian Holovaty <[EMAIL PROTECTED]> wrote: > > > After months of being frustrated (and hearing other people being > > frustrated) with our newish documentation index, I took some time > > tonight to reorganize the links, to make it easier and faster to find > > things. Take a look here: > > >http://docs.djangoproject.com/en/dev/ > > > I opted for a much more compact approach, with related links > > side-by-side. The previous version tended to organize each link based > > on whether it was reference vs. an overview, but this new version > > organizes by topic, instead. > > > The "Other batteries included" section is kind of a cop out. It's a > > bit past my bedtime, so that's all I could come up with in a hurry. > > > Hope people find this easier and faster to use! > > > Adrian > > -- > Clint Ecker --http://blog.clintecker.com > c: 312.863.9323 > --- > twitter: clint > skype: clintology > AIM: clintecker > Gtalk: [EMAIL PROTECTED] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Improving Errors from manage.py
Ahh. I suppose that makes sense. Is there any reason it doesn't show them by default? On Nov 17, 3:45 pm, "Alex Koshelev" <[EMAIL PROTECTED]> wrote: > Try to use `--traceback` switch that prints exception's traceback > > On Tue, Nov 18, 2008 at 00:41, David Cramer <[EMAIL PROTECTED]> wrote: > > > I've been trying to dump some data to fixtures for the last couple > > days, and I've ran into various problems each time. It's been quite > > difficult each time to determine the problems, as there's no useful > > tracebacks on the error messages: > > > [EMAIL PROTECTED]:~/Development/iplatform] ./manage.py dumpdata > > businesses > data > > Error: Unable to serialize database: Category matching query does not > > exist. > > > While this is great and all (it beats the "recursive error" that > > appeared in previous revisions with the same models), it would be much > > more useful if some of the commands would give more detailed > > information so that you can actually diagnose problems :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Improving Errors from manage.py
I've been trying to dump some data to fixtures for the last couple days, and I've ran into various problems each time. It's been quite difficult each time to determine the problems, as there's no useful tracebacks on the error messages: [EMAIL PROTECTED]:~/Development/iplatform] ./manage.py dumpdata businesses > data Error: Unable to serialize database: Category matching query does not exist. While this is great and all (it beats the "recursive error" that appeared in previous revisions with the same models), it would be much more useful if some of the commands would give more detailed information so that you can actually diagnose problems :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multi-Threaded Dev Server
I think I may have to actually agree that multi-threading is somewhat needed, since it does limit these kind of features from working. I, however, would deem it less a priority than most thing. If you can write a patch for it (or if one exists, I didn't look) though, I don't see any reason to not extend runserver to use it. On Nov 16, 6:45 am, Steve Holden <[EMAIL PROTECTED]> wrote: > Julian wrote: > > [...] I think some people want to use snippets like that [...] > > Wouldn't you agree that's a pretty feeble use case for something as > potentially disruptive as multi-threaded serving? Particularly when the > CherryPy alternative is so readily available. > > regards > Steve > -- > Steve Holden +1 571 484 6266 +1 800 494 3119 > Holden Web LLC http://www.holdenweb.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
I think forking it on github would be the best. I really like how easy it makes things. There's still quite a bit that will need done, mostly with code cleanup, and tests, but I think it will give everyone a pretty good start. If you'd like I can give you just direct commit access to the branch as well. On Sun, Nov 16, 2008 at 6:00 PM, Jacob Kaplan-Moss < [EMAIL PROTECTED]> wrote: > > On Sun, Nov 16, 2008 at 5:50 PM, David Cramer <[EMAIL PROTECTED]> wrote: > > http://github.com/dcramer/django-compositepks/tree/master# > > Cool; I'll take a look tomorrow. Do you want me to make changes in my > own tree, or would you rather patches back to you? > > Jacob > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
So far it allows declaration and creation of primary key models. You
declare them in class Meta: primary_key = ('field', 'field', 'field').
There is no ForeignKey/Relational? handlers as of right now, but the
admin is mostly working.
Alex Gaynor pointed out there is one unrelated change I accidently
committed. The list_select_related changes I proposed.
There is also an issue with URL resolving in the admin (its a 1.1
proposal) so it's using a , vs a / for the delimiter, but it's a
simply setting change.
On Nov 16, 5:50 pm, David Cramer <[EMAIL PROTECTED]> wrote:
> http://github.com/dcramer/django-compositepks/tree/master#
>
> On Nov 16, 5:25 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]>
> wrote:
>
> > On Sun, Nov 16, 2008 at 5:19 PM, David Cramer <[EMAIL PROTECTED]> wrote:
> > > Also keep in mind that work has been done, but as I haven't had a lot of
> > > spare time to continue it (attempt #3?). It's a very complex problem as
> > > well, like Jacob mentioned. You have to deal w/ the legacy use of single
> > > primary keys, as well as the new composites. While I almost have a fully
> > > functioning version (barring ForeignKey support) it's still going to take
> > > a
> > > little bit.
>
> > If you'll share your code — #373 is begging for a patch... — others
> > can use it as a starting place.
>
> > Jacob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
http://github.com/dcramer/django-compositepks/tree/master# On Nov 16, 5:25 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > On Sun, Nov 16, 2008 at 5:19 PM, David Cramer <[EMAIL PROTECTED]> wrote: > > Also keep in mind that work has been done, but as I haven't had a lot of > > spare time to continue it (attempt #3?). It's a very complex problem as > > well, like Jacob mentioned. You have to deal w/ the legacy use of single > > primary keys, as well as the new composites. While I almost have a fully > > functioning version (barring ForeignKey support) it's still going to take a > > little bit. > > If you'll share your code — #373 is begging for a patch... — others > can use it as a starting place. > > Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
Also keep in mind that work has been done, but as I haven't had a lot of spare time to continue it (attempt #3?). It's a very complex problem as well, like Jacob mentioned. You have to deal w/ the legacy use of single primary keys, as well as the new composites. While I almost have a fully functioning version (barring ForeignKey support) it's still going to take a little bit. On Sun, Nov 16, 2008 at 4:19 PM, Jacob Kaplan-Moss < [EMAIL PROTECTED]> wrote: > > Hi Frank -- > > It's hard for me to figure out how to answer this: if you've got a > problem with my leadership skills, I don't really see how anything I > say makes much of a difference. Frankly, your tone is completely > inappropriate and I feel I'm enforcing absurdly out-of-line behavior > simply by responding. > > However, I'm just going to ignore those parts of your post and focus > on the real question. > > Support for composite keys has indeed been requested before. In fact, > it's ticket #373; opened about three years ago! On July 20th, 2006, I > commented: > > """ > [He]re are the issues [...] that would need to be solved to make this work: > > There's three basic problems in dealing with composite primary keys in > Django. > > The first is that a number of APIs use "obj._meta.pk" to access the > primary key field (for example, to do "pk=whatever" lookups). A > composite PK implementation would need to emulate this in some way to > avoid breaking everything. > > Second, a number of things use (content_type_id, object_pk) tuples to > refer to some object -- look at the comment framework, or the admin > log API. Again, a composite PK system would need to somehow not break > this. > > Finally, there's the issue of admin URLs; they're of the form > "/app_label/module_name/pk/"; there would need to be a way to map URLs > to objects in the absence of a primary key. > """ > > (http://code.djangoproject.com/ticket/373#comment:3) > > That's a pretty clear list, and it's been sitting out there for over > two years. I've pointed folks at that comment any number of times > since then, and at some point someone expanded somewhat into a wiki > page (http://code.djangoproject.com/wiki/MultipleColumnPrimaryKeys) > that could serve as a simple spec. > > And yet three years on I've yet to see a patch appear on #373. Yes, > this gets asked for time and time again, but nobody seems to want it > enough to write even a *partial* fix. Why should this be? > > I think the main reason is that the lack of the feature is quite easy > to work around in most cases. So most people who could fix it just > don't feel like it's worth the time and move on. Somehow, despite the > strum und drang there isn't really enough energy here to prompt anyone > to work up a patch. > > Patches are the unit of currency in this community. With very few > exceptions, every one of Django's thousands of commits began life as a > patch posted by someone in the community. We committers can be a lot > more effective when we review and integrate other peoples' patches — I > can review a dozen patches from a dozen different people in the time > it takes me to fix one bug on my own — so by necessity we have to rely > on our community. > > If there's a feature you need, implement it. If you can't figure out > where to start, ask — I'm on #django-dev during most of the work week, > and I'd be happy to help anyone who wants to hack on this feature. If > you don't want to or can't implement it yourself, there's a legion of > options available ranging from asking around for help to organizing a > team to contracting someone qualified. > > Finally, please keep in mind that the feature list we're drafting > right now isn't set it stone. Anything that gets finished between now > and the feature freeze date for 1.1 (2/15/09) is a candidate for > inclusion. We develop these feature lists to help people figure out > what to work on; nobody's gonna tell anyone not to scratch their own > itch — that's what open source is all about. > > Jacob > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Multi-Threaded Dev Server
I'm a bit confused how multi-threading has anything to do with AJAX? Your requests will be slower, but they will still get processed (at least I've never had any issues). On Nov 16, 1:03 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > If the patch that's currently on there works with no changes at all, > I'd be hesitantly in favor of it, however if there any bugs, or other > complications associated with it, my answer is to use a real server, > setting up something like CherryPy locally is almost no effort, as > seen > here:http://www.oebfare.com/blog/2008/nov/03/writing-custom-management-com... > > On Nov 16, 1:39 am, "Calvin Spealman" <[EMAIL PROTECTED]> wrote: > > > Does it need to be multi-threaded? Can you get a multi-process dev > > server instead/also? > > > On Sun, Nov 16, 2008 at 1:26 AM, Chris <[EMAIL PROTECTED]> wrote: > > > > I thinkhttp://code.djangoproject.com/ticket/3357shouldbe given > > > another look at enabling optional multi-threading on the dev server. > > > > Jacob previously closed this ticket, noting the patch could introduce > > > threading bugs, and would provide functionality too similar to that of > > > a production environment. > > > > This is my rationale to accept this ticket: > > > 1. Ajax is very common place, and often requires a multi-threaded > > > server. The dev server is very convenient, but by not providing multi- > > > threading support, we're preventing it from being even more useful. > > > 2. Since multi-threading would be an optional setting, the default > > > scenario would still be single-threaded and would not break anything. > > > Only those specifically desiring multi-threading would subject > > > themselves to any unforeseen bugs. > > > 3. Fear of multi-threading bugs shouldn't be a reason to avoid multi- > > > threading, especially when it could be very useful. We don't even know > > > if there are multi-threading bugs. And even if there are, they can be > > > fixed. > > > > Regards, > > > Chris > > > -- > > Read my blog! I depend on your acceptance of my opinion! I am > > interesting!http://techblog.ironfroggy.com/ > > Follow me if you're into that sort of > > thing:http://www.twitter.com/ironfroggy --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django 1.1, app() and ticket #3591
I personally was a 0 on this one. Let me explain why. I want Django to be a strong platform for developers, like myself, who really want the opportunity to have power in the framework, as well as features. As of lately I have been using Rails for a project, and to be quite honest, the maturity and feature set of Rails makes Django a lot less fun. I think the app() (last I read) could be a nice addition, but I'd much rather see the more powerful features, which fix critical issues in Django, be put in first. Issues such as multiple database support, better URL resolving, and similar tickets. While this I think provides more flexibility for pluggables, and similar, I'd much rather see it wait til 1.2 (as the list of items in the 1.1 proposals is fairly massive). On Nov 15, 11:16 am, Michael Elsdörfer <[EMAIL PROTECTED]> wrote: > I haven't looked at the patch yet, but I'd really like to be able to > change an app's name (and with it the names of the database tables), > which I thought was something that this proposal would include. So > fwiw, I personally would like to see it in 1.1. > > Michael --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Branches on Github
Not the *best* place to post this, but it does relate to Django dev. Is it possible, and if so how, could one branch the Django trunk, and throw it on Github? >From what I've read I could then do some kind of "svn up" to merge in latest changes easily from trunk, and also work on my branch (e.g. composite primary keys). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
Ya this was more of a two birds one stone goal here. It'd be nice to be able to specify relations (e.g. the filter example), like it would with a generic foreign key, but not restricted to content_type/ object_id. We also will run into the issue with that, where object_id is a composite as well (but this can sit on the backburner IMHO). Anyways, adding the fields "transparently" (thats what I meant) is easy. I've done it quite a bit in my own code. I spent several hours however, trying to find a way to make ".filter(pk=(1,2,3))", ".filter (pk=dict(blah=1, blah2=2, blah3=3))" work, however. That is the specific area I've been waiting on. I have however had lots of time to find various random admin bugs which I've been slowly fixing up. Filters/etc are all working in my local version. On Nov 13, 10:14 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Thu, 2008-11-13 at 19:50 -0800, David Cramer wrote: > > I was thinking beyond just handling Foreign Keys actually when I wrote > > this up. It would allow for a more generic version of a generic > > foreign key. > > Generic relations are quite different to foreign keys and the two > shouldn't be merged. > > Generic Relations say how to get from an object id and a content type to > a remote object. They aren't a database-layer construct, although a > database constraint to, e.g., the content-type table could be > enforced(but that's not relevant here). ForeignKeys are a representation > of a database constraint, always referring to same remote table. They > are necessarily different concepts and I dont' think attempting to merge > them is going to gain much in clarity. At some point when multi-column > primary keys are supported, we might be able to work out a way for > generic relations to handle them, but I'm not holding my breath about > that and it's certainly a phase 2 or phase 3 thing. The reason it's > almost impossible to support generic relations with arbitrary > multi-column fields because the number of columns is, well, arbitrary > (attempting to jam multiple remote values into one local database column > is a bad idea, too. At some point the hacks mount up beyond the point of > usefulness). That is, one content type requires one value, a different > one requires two and a third content type might require four values to > identify the object. I strongly suspect we'll end up just saying > GenericRelation fields only work for single valued objects ids. Which > isn't at all unreasonable, since Django's generic relation classes > aren't really compulsory for data modelling. Any given situation has a > solution that doesn't involve them. > > > Although the more I think about it we'd still want to > > magically handle ForeignKey's so the same problem exists. > > There's no magic in programming. :-) > > Maybe be you mean transparently. If so, I think this is a requirement, > too, but it's not a problem. We know when the ForeignKey field is > specified which model it refers to, so we can include the correct number > of database columns at table creation time. We have perfect information > when we create the table, which is where this differs from any generic > relations. > > > I guess I could write a composite foreign key manager, which could be > > used in both instances. The problem is the API is very confusing at > > the moment, and I haven't been able to work out a good way to handle > > (in the code) mycomposite__exact=1 (single to multiple relation > > lookup). Maybe you could help with this :) > > The value on the right hand side can be any sequence or tuple (or even > an object that knows how to return a sequence or tuple). The ORM will > know that the field involved is a multicolumn field and know to expect > multiple values on the RHS to associate with the multiple LHS columns. > It doesn't make sense to say mycomposite__exact=1, since mycomposite > expects multiple values. It's clear what > > mycomposite__exact = (1,2) > > means, however. > > I will finish my work on this when I get a chance and as soon as > possible. I realise you've been waiting for a bit. Haven't had a lot of > time for Django the past few weeks, but I'm getting back on the horse > again now, although there'll be a bit of focus on 1.1 feature triage and > 1.0.X for a little bit first. > > It doesn't mean that you need to specify the primary key columns on the > ForeignKey field, though. As I pointed out in the original message, you > know which are the primary key columns because they're declared on the > model referred to. > > > >
Re: Feature reviews for 1.1
I volunteer for #17! And I have working code for a pretty simplistic message handler which is session vs database based (which is one of the proposals, sort of). So I guess if that's whats wanted, I can mold it to fit what has been discussed. On Nov 13, 5:03 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > On Thu, Nov 13, 2008 at 1:48 PM, Jacob Kaplan-Moss > > <[EMAIL PROTECTED]> wrote: > > I'd like to ask committers and anyone else to send me their own rankings. > > Also: There's no real need for everyone to score everything. In fact, > that'll probably be information overload. > > If you want to work on one of the features on this list, just let me > know; no need to rank 'em all. Remember that features without a > champion and implementor simply won't get done. So if you see one you > can do, volunteer to take it up! > > Jacob --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Composite Foreign Keys
I was thinking beyond just handling Foreign Keys actually when I wrote this up. It would allow for a more generic version of a generic foreign key. Although the more I think about it we'd still want to magically handle ForeignKey's so the same problem exists. I guess I could write a composite foreign key manager, which could be used in both instances. The problem is the API is very confusing at the moment, and I haven't been able to work out a good way to handle (in the code) mycomposite__exact=1 (single to multiple relation lookup). Maybe you could help with this :) Really, this issue is all thats stopping me from completing the composite PKs patch. On Nov 13, 7:19 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Thu, 2008-11-13 at 10:53 -0800, David Cramer wrote: > > Here's my proposal for the composite keys (which act much like generic > > keys except more useful): > > > class MyModel2: > > pass > > > class MyModel3: > > pk_part1 = models.AutoField() > > pk_part2 = models.ForeignKey(MyModel2) > > name = models.TextField() > > > class MyModel: > > pk_part1 = models.IntegerField() > > pk_part2 = models.ForeignKey(MyModel2) > > > mymodel3 = models.CompositeForeignKey(MyModel3, pk_part1=pk_part1, > > pk_part2=pk_part2) > > Why is the primary key specified on the relation field, rather than just > being part of MyModel3? A model has a primary key, and the foreign > relation should use that in the natural case. It's not a property of the > relation, it's a property of the target model and should be declared > there. > > There might be an argument for a multi-column version of to-field, but a > tuple or list would seem the easiest way of specifying that, rather than > a variable number of arguments. > > At that point, I think normal ForeignKey fields could be adjusted to > Just Work(tm), since they'll know they're related to a model with a > multi-column primary key. > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Proposal: Composite Foreign Keys
Here's my proposal for the composite keys (which act much like generic keys except more useful): class MyModel2: pass class MyModel3: pk_part1 = models.AutoField() pk_part2 = models.ForeignKey(MyModel2) name = models.TextField() class MyModel: pk_part1 = models.IntegerField() pk_part2 = models.ForeignKey(MyModel2) mymodel3 = models.CompositeForeignKey(MyModel3, pk_part1=pk_part1, pk_part2=pk_part2) MyModel.objects.filter(mymodel3__name="Hi") Anyways, I think you get the gist. I think this would cover all use cases, and is a very simple, similar, and clean API. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Ordering in the admin and via the model API
Ya this is much more needed for me on ForeignKey's, but that's like adding magical models (which I like!). You'd have to add left order, and right order fields, on the model referenced in the ForeignKey, to have an OrderedForeignKey field. This, without migrations, would not be fun with DB maintenance :) On Nov 12, 7:23 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > I'd just like to point out that this isn't something to solve just for > many to many's, it's also for ForeignKeys. > > On Nov 12, 6:58 pm, Jonas Pfeil <[EMAIL PROTECTED]> wrote: > > > The idea of an ordered many-to-many field is very old [1] and has been > > proposed a lot over the years (e.g. [2], [3]). So the general > > consensus seems to be that we want such a thing. I’d like to share > > some thoughts on how to approach the whole ordering problem on the API > > side (leaving aside the JavaScript implementation issues for a > > moment). > > > I can think of three things we would want an ordering UI for in the > > admin: > > > * A yet to be devised OrderedManyToMany field > > * A ManyToMany field with an intermediary model > > * Ordinary models (in the change list) > > > OrderedManyToMany > > = > > > We could either subclass ManyToMany or introduce another option (like > > ordered=True). I think subclassing is better, because of the extra > > functions we could add to manipulate the many-to-many set. Like: > > > * move_up/down(item) – Moves the item one slot up or down in the list > > * swap(a, b) – Swaps the places of item a and b > > * insert(item, position) – Puts the item at the stated position, > > pushing the other items down (or we just add a position argument to add > > () and create()) > > * push/pop() – Or just add/create() without position argument and > > remove() with no arguments at all > > > Of course the existing functions like all(), assignment of a list to > > the field and so on should all respect the order too. > > > Many-to-many fields with intermediary models should also return the > > correct order when reading the set, even if they do not support create > > (), add() and assignment of a list. > > > Ordinary models and ManyToMany fields with an intermediary model > > == > > > These two use cases share the need to define the field which should be > > used for ordering. I found it difficult to come up with a name that > > can’t be confused with 'ordering' too easily. How about this: > > > class Membership(models.Model): > > order = models.IntegerField() > > > class Meta: > > order_control_field = ‘order’ > > > On a side note: When the change list in the admin is ordered on > > something else than the order_control_field, an ordering UI stops > > making sense (see description on [4]). A solution would be to just > > switch off ordering on other fields if order_control_field is set. For > > inlines this problem is nonexistent as the admin currently doesn’t > > reorder the inlined items when you click the column header. > > > Ok, these are my ideas on how ordering could work for the user. What > > do people think? > > > Cheers, > > > Jonas > > > [1]http://code.djangoproject.com/ticket/13 > > [2]http://code.djangoproject.com/ticket/6118 > > [3]http://groups.google.am/group/django-developers/browse_thread/thread/... > > [4]http://www.djangosnippets.org/snippets/998/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Make filters in admin persistent (#6903)
I was implying that if I'm in email admin, and there's filters there, once I leave email admin, there's no reason to keep that information around. Going back to the email admin should give a clean start, that's how typical applications like that work (at least every one that I've ever used). But yes, you'd want to preserve any GET params most likely. Easiest to just save the URL. On Nov 11, 4:39 pm, George Vilches <[EMAIL PROTECTED]> wrote: > I wasn't thinking that the filters would be preserved across > *different* models or admin pages, only within them. So, you'd keep > some sort of dictionary, keyed based on the particular admin URL, > model, or some other easily achievable unique piece of information per > screen. I wouldn't expect that just because I was filtering on e-mail > address in one section of the admin app, it would automagically apply > that filter wherever else it could. :) > > Would preserving filters open the door to also preserving the current > ordering and current page number in the admin view? I would suspect > that if you're saving one, you'd really want to save all three, > because they all go together insofar as returning you to *exactly* the > view you were last using. If you're going to preserve it at all, > might as well do it right and preserve it as pristinely as possible. > > On Nov 11, 2008, at 5:34 PM, David Cramer wrote: > > > > > I guess you're right. This is just for admin so it's not a huge deal. > > > It will feel weird howerver, if I somehow go to a search results page > > and it remembers my filters when I was visiting something else before > > that. So it'd need to handle clearing it at the right time as well. > > > On Nov 11, 3:51 pm, George Vilches <[EMAIL PROTECTED]> wrote: > >> Don't sessions already have standard expirations in them? Besides > >> that, this is the admin, it's not a tool for every user of your > >> application to be in, so there will only be a few larger sessions > >> (and > >> larger is still only a few K at most, if you have lots and lots of > >> models you're filtering on). And yes, it would only keep it for the > >> length of that user's session, I don't magically expect it to be able > >> to suddenly transfer to another user. If I wanted that > >> functionality, > >> I would ask for something to be added to the admin that would print > >> out a URL that you could give to another user to get the filtering > >> you > >> were just using. Which sounds handy, but is a separate ticket from > >> what we're discussing here. > > >> On Nov 11, 2008, at 4:47 PM, David Cramer wrote: > > >>> Well I'm not sure storing multiple search paths is too good of an > >>> idea, as you increase the size of the session significantly, and > >>> then > >>> have to worry about expiring those or clearing them somehow. The > >>> session just keeps it for that users session, vs whoever else > >>> happens > >>> to visit that url (say I pass it off to a coworker). > > >>> On Nov 11, 3:39 pm, George Vilches <[EMAIL PROTECTED]> wrote: > >>>> I definitely second this. The extra bonus to storing it in the > >>>> session is that you can maintain your search state on multiple > >>>> admin > >>>> pages/models independently without overflowing the URL. > >>>> Naturally if > >>>> you do it this way, you'd also want to have a visible "clear > >>>> filters" > >>>> link so that there's some way to reset that state, I didn't check > >>>> the > >>>> patch to see if this was already included. > > >>>> On Nov 11, 2008, at 4:35 PM, David Cramer wrote: > > >>>>> Before this gets accepted, I'd like to throw in the proposal of > >>>>> storing this in the session vs a huge URL. That and a hash seem to > >>>>> be > >>>>> the common approach to storing search paths. > > >>>>> On Nov 11, 7:19 am, Jonas Pfeil <[EMAIL PROTECTED]> wrote: > >>>>>> Currently if you search in the admin, use some kind of filter or > >>>>>> even > >>>>>> just go to the second page in the change list this selection is > >>>>>> reset > >>>>>> when you edit an item and hit save. The user gets the default > >>>>>> list > >>>>>> again. Needless to say this can be quite annoying. Especially if > >>>>>> you > >>>>>> want to edit a specific subset of a very large database. > > >>>>>> The solution is to somehow make the filters persistent. The > >>>>>> ticket > >>>>>> [1] > >>>>>> already has a patch. > > >>>>>> Cheers, > > >>>>>> Jonas > > >>>>>> [1]http://code.djangoproject.com/ticket/6903 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Make filters in admin persistent (#6903)
I guess you're right. This is just for admin so it's not a huge deal. It will feel weird howerver, if I somehow go to a search results page and it remembers my filters when I was visiting something else before that. So it'd need to handle clearing it at the right time as well. On Nov 11, 3:51 pm, George Vilches <[EMAIL PROTECTED]> wrote: > Don't sessions already have standard expirations in them? Besides > that, this is the admin, it's not a tool for every user of your > application to be in, so there will only be a few larger sessions (and > larger is still only a few K at most, if you have lots and lots of > models you're filtering on). And yes, it would only keep it for the > length of that user's session, I don't magically expect it to be able > to suddenly transfer to another user. If I wanted that functionality, > I would ask for something to be added to the admin that would print > out a URL that you could give to another user to get the filtering you > were just using. Which sounds handy, but is a separate ticket from > what we're discussing here. > > On Nov 11, 2008, at 4:47 PM, David Cramer wrote: > > > > > Well I'm not sure storing multiple search paths is too good of an > > idea, as you increase the size of the session significantly, and then > > have to worry about expiring those or clearing them somehow. The > > session just keeps it for that users session, vs whoever else happens > > to visit that url (say I pass it off to a coworker). > > > On Nov 11, 3:39 pm, George Vilches <[EMAIL PROTECTED]> wrote: > >> I definitely second this. The extra bonus to storing it in the > >> session is that you can maintain your search state on multiple admin > >> pages/models independently without overflowing the URL. Naturally if > >> you do it this way, you'd also want to have a visible "clear filters" > >> link so that there's some way to reset that state, I didn't check the > >> patch to see if this was already included. > > >> On Nov 11, 2008, at 4:35 PM, David Cramer wrote: > > >>> Before this gets accepted, I'd like to throw in the proposal of > >>> storing this in the session vs a huge URL. That and a hash seem to > >>> be > >>> the common approach to storing search paths. > > >>> On Nov 11, 7:19 am, Jonas Pfeil <[EMAIL PROTECTED]> wrote: > >>>> Currently if you search in the admin, use some kind of filter or > >>>> even > >>>> just go to the second page in the change list this selection is > >>>> reset > >>>> when you edit an item and hit save. The user gets the default list > >>>> again. Needless to say this can be quite annoying. Especially if > >>>> you > >>>> want to edit a specific subset of a very large database. > > >>>> The solution is to somehow make the filters persistent. The ticket > >>>> [1] > >>>> already has a patch. > > >>>> Cheers, > > >>>> Jonas > > >>>> [1]http://code.djangoproject.com/ticket/6903 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Make filters in admin persistent (#6903)
Well I'm not sure storing multiple search paths is too good of an idea, as you increase the size of the session significantly, and then have to worry about expiring those or clearing them somehow. The session just keeps it for that users session, vs whoever else happens to visit that url (say I pass it off to a coworker). On Nov 11, 3:39 pm, George Vilches <[EMAIL PROTECTED]> wrote: > I definitely second this. The extra bonus to storing it in the > session is that you can maintain your search state on multiple admin > pages/models independently without overflowing the URL. Naturally if > you do it this way, you'd also want to have a visible "clear filters" > link so that there's some way to reset that state, I didn't check the > patch to see if this was already included. > > On Nov 11, 2008, at 4:35 PM, David Cramer wrote: > > > > > Before this gets accepted, I'd like to throw in the proposal of > > storing this in the session vs a huge URL. That and a hash seem to be > > the common approach to storing search paths. > > > On Nov 11, 7:19 am, Jonas Pfeil <[EMAIL PROTECTED]> wrote: > >> Currently if you search in the admin, use some kind of filter or even > >> just go to the second page in the change list this selection is reset > >> when you edit an item and hit save. The user gets the default list > >> again. Needless to say this can be quite annoying. Especially if you > >> want to edit a specific subset of a very large database. > > >> The solution is to somehow make the filters persistent. The ticket > >> [1] > >> already has a patch. > > >> Cheers, > > >> Jonas > > >> [1]http://code.djangoproject.com/ticket/6903 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Proposal: Make filters in admin persistent (#6903)
Before this gets accepted, I'd like to throw in the proposal of storing this in the session vs a huge URL. That and a hash seem to be the common approach to storing search paths. On Nov 11, 7:19 am, Jonas Pfeil <[EMAIL PROTECTED]> wrote: > Currently if you search in the admin, use some kind of filter or even > just go to the second page in the change list this selection is reset > when you edit an item and hit save. The user gets the default list > again. Needless to say this can be quite annoying. Especially if you > want to edit a specific subset of a very large database. > > The solution is to somehow make the filters persistent. The ticket [1] > already has a patch. > > Cheers, > > Jonas > > [1]http://code.djangoproject.com/ticket/6903 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: select_related optimization and enhancement for the django.contrib.admin
This change is here because when you say "this foreign key is in list_filter" it immediately does a select_related() (grabbing every single relation which is ridiculous). This change says "only grab the foreign keys which are used. The only addition is that list_select_related can be a boolean, or a list. This, in my eyes, is fully backwards compatible, as its only an addition to the public API, and a bug fix. On Tue, Nov 11, 2008 at 3:17 AM, Malcolm Tredinnick < [EMAIL PROTECTED]> wrote: > > > On Mon, 2008-11-10 at 17:13 -0800, David Cramer wrote: > [...] > > Anyways, what it does: > > > > * list_select_related can be a boolean, or a list. If it's a list it > > says "select_related on these fields" > > How is that different functionality from just specifying the names of > the fields in the current API? Are you just proposing a different API to > achieve the same thing, or is there an addition here? > > > * select_related is smart by default, no more implicit select all > > Backwards incompatible and I don't really see the need for it (can you > motivate why you're proposing this change?). Strong -1 for those reasons > (particularly the backwards incompatibility). > > Regards, > Malcolm > > > > > > -- David Cramer Director of Technology iBegin http://www.ibegin.com/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Denormalisation Magic, Round Two
I'm not sure on AggreateField either. What if you just do like
("Photo", "user__exact=self.user") or something. Currently there's no
rerepsentation for "self" in Django QuerySet's, so this is a hard
thing to call. Also, we need a way to support callables.
e.g. models.MirrorField("my_callable_or_attribute") as well as the
models.MirrorFIeld("my_attribute", "its_subattribute")
On Oct 27, 6:26 am, Andrew Godwin <[EMAIL PROTECTED]> wrote:
> Russell Keith-Magee wrote:
> > I do have a slight reservation regarding the API you are proposing.
> > The fields you have proposed (MirrorField and AggregateField) capture
> > the obvious use cases, but the syntax don't feel consistent with the
> > rest of the Django API. In particular, AggregateField seems to be
> > introducing a whole new query syntax that is completely independent of
> > the aggregation syntax that is due for inclusion in v1.1.
>
> > I would prefer to see a syntax that demonstrated slightly better
> > integration with the Django query syntax. If it isn't possible to use
> > QuerySet itself as a way to develop the underlying query, then I would
> > expect to see a syntax that at the very least retained the flavour of
> > the Django query syntax rather than inventing "fk:" style operators
> > that have no real analog.
>
> Indeed; I was far happier with the queryset syntax for AggregateField I
> originally proposed, but I couldn't get it to work - it just won't go
> inside class bodies without far, far too much hackery. The MirrorField
> API I like more; I have an alternate version where you simply pass it
> the model name and column name, but that then has issues if you have
> more than one FK to the same model.
>
> The AggregateField proposal is just that, and I really squirmed at the
> idea of "all:" and "fk:", but it got it working pretty quickly. One
> possibility for more clean integration is an approach more like
> querysets, so you could have something like
>
> photos_count = models.AggregateField(Photo.view.filter(user_self=True))
>
> The main problem with anything like this (and indeed with my attempt to
> implement this with plain old querysets) is that, as a field, you exist
> only on a class, and so when you get a signal saying a model has been
> updated, it's somewhat difficult to determine which instances of
> yourself to update - you can't loop through them all testing, since
> that's hilariously inefficient. That's why I limited the querying to
> only fk: and all: types, since detecting these is far more efficient.
>
> I'd love to see any proposal for how Aggregates should look, as my
> current incarnation really isn't too django-ish. I personally like
> MirrorFields (now on their third name, more welcomed) as I have them
> now, but then I would, since I invented them for my own use...
>
> Andrew
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
order_by_fields for django.contrib.admin
One of the last of my feature-needs (for optimization reasons) is the
ability to specify what fields can, or can't be ordered in the admin
apps.
I'd like to propose something along the lines of:
order_by_fields = ('field', 'field', 'field')
This would allow you to only order by fields which you've created
indexes on.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
select_related optimization and enhancement for the django.contrib.admin
While I've been working on composite primary keys, I also made a few tweaks to the admin app. Most of these are related to the primary key support, but one is an optimization. I would post this as a patch, or a ticket, but I want to open this up for discussion, and my django repo is so far from trunk it's a PITA at the moment to create the patch. Anyways, what it does: * list_select_related can be a boolean, or a list. If it's a list it says "select_related on these fields" * select_related is smart by default, no more implicit select all If someone wants to create a patch, you'll be able to get to it a lot faster than I will. Otherwise I'll eventually throw one up on trac for each change. line 198ish of django.contrib.admin.views.main if isinstance(self.list_select_related, (tuple, list)): qs = qs.select_related(*self.list_select_related) elif self.list_select_related: qs = qs.select_related() else: fields = [] for field_name in self.list_display: try: f = self.lookup_opts.get_field(field_name) except models.FieldDoesNotExist: pass else: if isinstance(f.rel, models.ManyToOneRel): fields.append(name) if fields: qs = qs.select_related(*fields) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Cache and GET parameters
I really like the idea of the explicit GET params passed.So I'm +1 especially on solution #3. I actually had never realized it wasn't caching pages with GET params, luckily though, any pages where I use this decorator don't fluctuate like that :) On Nov 1, 7:51 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > On Sat, Nov 1, 2008 at 8:32 PM, SmileyChris <[EMAIL PROTECTED]> wrote: > > > On Nov 2, 2:52 am, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > >> Assuming vary_on_get() with no parameters means no variance (other > >> than the HTTP Vary headers), then [...] > > > That seems confusing - the decorator name seems to imply that it would > > vary on any get attribute (even though this is the default) - at least > > that's how I'd look at it if I didn't know otherwise. > > @vary_on_get(None) ? :-) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
