Yes! I'm in. Out of curiosity: When inserting lots of data, how do you do it? Using the orm? Have you looked at http://pypi.python.org/pypi/dse/2.1.0 ? I wrote DSE to solve inserting/updating huge sets of data, but if there's a better way to do it that would be especially interesting to hear more about ( and sorry for the self promotion ).
Regards, Thomas On Wed, Jun 22, 2011 at 3:15 PM, Cal Leeming [Simplicity Media Ltd] <cal.leem...@simplicitymedialtd.co.uk> wrote: > Hi all, > Some of you may have noticed, in the last few months I've done quite a few > posts/snippets about handling large data sets in Django. At the end of this > month (after what seems like a lifetime of trial and error), we're finally > going to be releasing a new site which holds around 40mil+ rows of data, > grows by about 300-500k rows each day, handles 5GB of uploads per day, and > can handle around 1024 requests per second on stress test on a moderately > spec'd server. > As the entire thing is written in Django (and a bunch of other open source > products), I'd really like to give something back to the community. (stack > incls Celery/RabbitMQ/Sphinx SE/PYQuery/Percona > MySQL/NGINX/supervisord/debian etc) > Therefore, I'd like to see if there would be any interest in webcast in > which I would explain how we handle such large amounts of data, the trial > and error processes we went through, some really neat tricks we've done to > avoid bottlenecks, our own approach to smart content filtering, and some of > the valuable lessons we have learned. The webcast would be completely free > of charge, last a couple of hours (with a short break) and anyone can > attend. I'd also offer up a Q&A session at the end. > If you're interested, please reply on-list so others can see. > Thanks > Cal > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-users@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > -- Mvh/Best regards, Thomas Weholt http://www.weholt.org -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
