How to use the MigrationAutodetector manually for dynamic models

[Diederik van der Boor]

Hi All,

I'm working on an API-server project that uses dynamically generated Django 
models, so it can service many user-provided datasets. To ease maintenance 
work, we'd like to generate the SQL statements to migrate database tables 
to a newer version of the schema. I've managed to use the internal Django 
migration API's for this.

Unfortunately, the call to migration.apply(old_state, schema_editor, 
collect_sql=True) breaks when there is more then one operation in the 
migration object. The ProjectState model fields get bound to a model, and 
therefore the next operation won't run.

Is there anything we're doing wrong? Or maybe we're using this migration 
API in ways that aren't foreseen in its design, but can be worked around?

(code is here in a branch: 
https://github.com/Amsterdam/schema-tools/blob/diederik/sqlmigrate-schema/src/schematools/contrib/django/management/commands/sqlmigrate_schema.py)

Best,
Diederik

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/9c54b72b-7c3b-4f93-b9db-e77e77b5925dn%40googlegroups.com.

Re: mass email app for django

[Diederik van der Boor]

Hi Sven,

What you describe sounds like a Mailinglist..
Wouldn't it be possible to use Mailchimp or Aweber for this kind of service?
>From Django you can use their API's.

Mass-mailing has a risk of being blacklisted as server,
at least make sure you send email via a whitelisted SMPT server (and not 
directly to everyone at the web).
For mass-mailing I always recommend using specialized mailinglist software.

Greetings,
Diederik

Op 15 jan. 2014, om 01:42 heeft Sven Aßmann  het volgende 
geschreven:

> Hi Group,
> 
> currently i evaluating / research for a django app that supports mailing 
> specific texts to a receipient list (like mass emailing). the corner features 
> are:
> 
> - email text should be editable (text or html nothing special) thru the admin 
> app
> - the receipient list should be maintainable independently from users and 
> texts (like a addressbook that is maintanale thru admin app)
> - ideally supports several mail backends like smtp, aws sms, postmark etc..
> - processing of the sending can be done async via django cli / message queue 
> services 
> 
> The closest use case app would be something like a newsletter app, where the 
> subscibers can be maintainted apart from django.auth.users
> I am aware of www.djangopackages.com and found already a couple of 
> applications that could make the job.
> 
> I ask you folks if someone has good experience with a specific one and or 
> what 
> features are included and helpfull and what are not that helpfull. 
> 
> I really appreciate your contributions.
> 
> -- 
> 
> 
> Best Regards, 
> Sven
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/3688706.24zruyElKd%40gilgamesch.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/2A731BC5-4F93-433F-9E04-5267DDC82719%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: What is the right location for my django translation files?

[Diederik van der Boor]

You can place a new .po file in a `locale` folder of one of your INSTALLED_APPS.
This will be picked up, and can override the locale of mezzanine,
if your app is placed before mezzanine in INSTALLED_APPS.

Off course, run django-admin.py compilemessages in the root folder where the 
`locale` folder exists,
so the .mo file also exists and restart the runserver afterwards.

Greetings,
Diederik


Op 9 okt. 2013, om 08:41 heeft Anton Melser  het 
volgende geschreven:

> I'm giving my first steps with python/django/mezzanine so please bear with me.
> I modified a translation file of the mezzanine's blog application and 
> compiled it OK. Mind you: I literally only modified a couple of words and 
> left the rest intact.
> 
> The only place I found out I could place them in order to test was at the 
> blog app's locale folder 
> (~/MY_VIRTUAL_ENV/lib/python2.7/site-packages/mezzanine/blog/locale/es/LC_MESSAGES).
> 
> It worked fine, but my guts tell me there has to be a better way so I can 
> have this file(s) in some other location WITHIN my own mezzanine application 
> so:
> 
> I can easily maintain them and
> I don't have to keep my whole virtual environment in my SCM's repository.in 
> order to keep track of this single file with just a couple or words modified 
> and that will hardly ever get modified again.
> Great minds think alike? :-). I am in *exactly* the same situation - there 
> are a couple of translations I would like to change slightly in Mezzanine (a 
> capitalisation here, pluralisation there) and I had exactly the same thought. 
> I don't want to have to maintain something completely separately. Did you 
> find an elegant solution?
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/80e29974-637f-49d4-83e8-f176829cb18e%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/5568572E-18F0-4147-AE21-B6C5BAE54598%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Using Django RelatedField for custom join queries?

[Diederik van der Boor]

Hi all,

I've asked this question on stackoverflow 
(http://stackoverflow.com/questions/18913661/using-django-relatedfield-for-custom-join-queries),
but figured it might be better to post it here:

I'm curious whether a RelatedField can be used to cause custom join queries.
I would like to apply this in django-parler 
(https://github.com/edoburu/django-parler), a multilingual app for Django.

For example, when "slug" is a translated field, I would like to have:

MyModel.objects.filter(slug="foo")

to operate as:

MyModel.objects.fiter(translations__slug="foo")

under the hood. This is possible with the API of the RelatedField class for 
example?

I noticed there are several API hooks in the RelatedField class, such as 
m2m_reverse_field_name(), m2m_target_field_name(), m2m_column_name(), 
extra_filters() etc.. which are used in the GenericRelation and django-taggit's 
TaggableManager classes but I have no idea how these work.

N.B. Regular field access on instances is covered in django-parler (using 
descriptors that proxy to calls to the translated table), and I would love to 
extend support to ORM queries as well.

Best regards,
Diederik

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
For more options, visit https://groups.google.com/groups/opt_out.

Re: OperationalError: (2003, "Can't connect to MySQL server on 'localhost' (10061)")

[Diederik van der Boor]

The hostname "localhost" has a special meaning for MySQL, at least on Linux it 
has.
When you ask the mysql driver to connect to localhost, it will use the socket.
To avoid this issue, better use 127.0.0.1.

Also check whether the MySQL server is running on that port (netstat -a ?),
whether the service is up (see services panel),
and whether your Windows Firewall allows connections to it.

Greetings,
Diederik

Op 6 jan. 2013, om 18:28 heeft Александр Соколов  
het volgende geschreven:

> Hi! Recently I had the same problem and this site has helped me: 
> http://stackoverflow.com/questions/11121819/mysqldb-in-python-cant-connect-to-mysql-server-on-localhost
> I've set the next properties:
> 'ENGINE': 'django.db.backends.mysql', 
> 'NAME': 'yourdbname',  
> 'USER': 'root',  
> 'PASSWORD': 'your password', 
> 'HOST': '127.0.0.1', 
> 'PORT': '3306', 
> 
> Hope, it will help you.
> 
> P.S. By the way in the beginning, I've installed the latest version of 
> MySQLdb. But this is unlikely has been causing of my problem.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/django-users/-/dFs9n5ZCuDEJ.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Postgres versus mysql? Or, is RDS worth migrating for?

[Diederik van der Boor]

Second that. If you consider moving back to MySQL,
at least make they've seen the following video at:

http://wekeroad.com/2012/07/19/postgresql-rising

;-)

Secondly, timezones of 1.4 are painful in MySQL to convert to,
and south migrations won't be able to rollback failed schema changes.

Greetings,
Diederik

Op 5 nov. 2012, om 16:44 heeft Faraz Masood Khan het volgende geschreven:

> I recommend if you do transactions a lot, stick to Postgresql by all means. 
> MySQL isn't transactional database, to test what I mean, create a InnoDB 
> database with a very simple InnoDB table (two columns would be fine), than do 
> 1000 inserts in transactions; note the time and you will find that InnoDB is 
> very very slow on insert, plus transactions death locks are very frequent.
> 
> On Monday, November 5, 2012 7:50:56 AM UTC+5, martharotter wrote:
> I was advised recently to port my startup's Django databases from postgres to 
> mysql for two reasons: 
> 
> 1) RDS on Amazon makes it your life much easier
> 2) When I need to scale my web app and have to hire people to help me, it 
> will be much easier to find solid mysql people than postgres people. 
> 
> Any thoughts on these? Are they legitimate concerns? It's always a pain to 
> migrate, but I guess if it's worth doing I'd rather do that now when it makes 
> less of an impact. 
> 
> Thanks in advance for your help!
> --Martha 
> http://woop.ie 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/django-users/-/dJ8jP6DRylgJ.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Django mod-wsgi multiple instances of One Class object

[Diederik van der Boor]

In pre 1.4 projects, it was common for projects to have multiple ways to import 
stuff;
because the project path, and the path 1 level up were both in the python path.

This way, you can actually have 2 versions of the module imported,
and hence your singleton breaks (depending on the route taken to access the 
object).

Please make sure your imports are consistent, and your `sys.path` is clean.

Greetings,
Diederik


Op 5 nov. 2012, om 19:07 heeft drunksaint het volgende geschreven:

> I am currently using a DJango - apache installation (on mod_wsgi) to run a 
> website (project) which hosts an intranet cards game, whose rules are 
> complicated enogh that it requires only 1 master list+dict data structure of 
> scores for each user. Individual user's score can be impacted by other users 
> actions also. I am using a Score class which I instantiate in urls.py and all 
> players / users access this class' object [scObj = Scorer('Spades') ]. 
> internally in this obj there are Dicts of players actions, score updates etc 
> etc.
> 
> class Scorer:
> def __init__(self, suitname):
> self.__suitname = suitname
> The scorer class is separately stored in Scorer.py
> 
> scObjs = (Scorer('Spades'), Scorer('Hearts'), Scorer('Clubs'), 
> Scorer('Diamonds'),)
> The objects are created in urls.py
> 
> This model was running perfectly in the DEV deployement. When I moved to PROD 
> in apache, this broke (as in there are multiple instances of scObj being 
> created, so a score update from player-1's move on player-2 is not reflected 
> in a different move by player-3 on player-2) . I need to simulate a singleton 
> class behaviour / global scObj behaviour, which I am not able to do. I am 
> running apache in worker - mpm - multithreaded mode, so I assume that the 
> problem of multiple copies of scObj being there due to multi-process is not 
> there.
> 
> Is the problem related to mod_wsgi?
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/django-users/-/omNsLiRhcpkJ.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Model field that links to a page ID, or external URL

[Diederik van der Boor]

Op 13 feb. 2012, om 18:50 heeft Jonathan Paugh het volgende geschreven:

> Each model supports a get_absolute_url(self) method, which allows you to
> retrieve each object's url in whatever way you want, including pulling
> values from self (i.e. the model object.). Returned urls are in serveral
> formats; my prefered is '/full/path/to/article'.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-users?hl=en.
> 

I'm reluctant to use such method, when the absolute URL changes, the links 
break.
With a "pageid://##" solution I dynamically get get_absolute_url() when it's 
read.

Also, this solution gives me another problem: the URL needs to be reversed to 
find the ID
which was used in the select box / raw_id_field to choose the model.

if the solution still involves storing the value in a string field,
I still prefer my solution of storing the ID over a copied URL.

Diederik

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Model field that links to a page ID, or external URL

[Diederik van der Boor]

Hi,

In CMS interfaces I generally encounter a problem with URL fields.
In most situations, the following options need to be supported:
- An URL to an external page  (Django's URLField can handle that)
- An URL to an internal page (e.g. Django CMS)
- An URL to an other model (e.g. article)

How can I reasonably store this information in a model field?
I would love to store these values as foreign key, as it would allow more 
querying options.

I've written a solution for this before using a forms.MultiValueField / 
MultiWidget input,
which did involve the ugly hack of saving the contents as string, using the 
format:
- http://...external-page/
- pageid://##-- the internal page
- articleid://##  -- link to an article.
This works, and did fit in the project deadline.
However, I would like to improve upon this.

Does anyone have a suggestion on how to store the various information properly?

Greetings,
Diederik

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Django Web Application Cost So Many memory

[Diederik van der Boor]

Op woensdag 30 maart 2011 17:48:38 schreef 付毅:
> hi Xavier Ordoquy
> So , you mean 60M per python process is in a normal status? I really
> encouter this kind of website before

Please note this is for the entire website, not per apache instance.
There is one Django process (via mod_wsgi, or standalone fastcgi) that
handles all requests coming from visitors.

I hope this was obvious, but I got the impression that bit got lost in the 
discussion,

Greetings,
Diederik

 
> On Wed, Mar 30, 2011 at 11:35 PM, Xavier Ordoquy 
wrote:
> > Le 30 mars 2011 à 15:41, 付毅 a écrit :
> > > i don't think 100 instance means to 100 CPUs. I just want to use
> > 
> > http_load to test pressure to my web server. if i make the 100 cocurrent
> > request per second with 100 cocurrent connection. I will make 100 python
> > process instance when I deploy my web project in prefork environment. Any
> > others agree with me?
> > 
> > If you have 100 instances, 9 of 10 which are waiting in python code,
> > you've got a huge architecture issue.
> > Let a front server (nginx, apache, and co) deal with administrative tasks
> > and let django deal with computing the pages.
> > There are tons of blog posts out there that explains how to do it.
> > 
> > Regards,
> > Xavier.
> > 
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django users" group.
> > To post to this group, send email to django-users@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-users+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Hosting many small Django websites

[Diederik van der Boor]

Thanks for your suggestion about Opus, will look into that.
The security concerns are a good argument for running separate wsgi processes 
afterall.

Diederik

Op Friday 15 October 2010 00:34 schreef Brian Bouterse:
> We host many of our django sites using a django deployer we wrote called
> Opus <http://github.com/bmbouter/opus>.  Opus uses a separate wsgi daemon
> process for each site.  Each wsgi process also runs a separate user, so
> linux would prevent the memory from being shared.  In our case though we
> want to run code from many different sources and if they all ran as the
> same user, then one django project could monkey patch another one which
> creates all kinds of interesting security issues.
> 
> Brian
> 
> On Thu, Oct 14, 2010 at 4:45 PM, Diederik van der Boor 
<vdb...@gmail.com>wrote:
> > Hi,
> > 
> > I'm curious about using Django for many small web sites. Does this
> > require each site to run in a separate wsgi daemon process? If so, how
> > is it possible to keep memory usage shared?
> > 
> > Thanks in advance,
> > 
> > Diederik
> > 
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django users" group.
> > To post to this group, send email to django-us...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-users+unsubscr...@googlegroups.com<django-users%2bunsubscr...@goog
> > legroups.com> .
> > For more options, visit this group at
> > http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Hosting many small Django websites

[Diederik van der Boor]

Hi,

I'm curious about using Django for many small web sites. Does this
require each site to run in a separate wsgi daemon process? If so, how
is it possible to keep memory usage shared?

Thanks in advance,

Diederik

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Where do you put your business logic in django? Organizing big projects in django.

[Diederik van der Boor]

Op maandag 27 september 2010 09:08:53 schreef MrMuffin:
> Where do you put your business logic in django? In my project I`ve put
> it into the models.py, but that file soon become huge and hard to
> maintain. Of course I can just stuff it into whatever file I like, but
> I`d like to have some standard way of doing this. There seems to be
> something missing in django when it comes to business logic. It`s all
> model, views and templates and that`s all great for small projects,
> but both the models.py and views.py very soon gets huge and how do you
> re-organize your project when that happens? Splitting views and models
> into seperate files is only a partial solution and that requires some
> hackish code in __init__.py to make syncdb etc work. And then there`s
> urls.py.

I'm surprised no one mentioned manager classes yet. This is an example for 
models.py:

  from django.db import models
  from yourapp.managers import SomeObjectManager

  class SomeObject(models.Model):
objects = SomeObjectManager()


Now you can do things like: SomeObject.objects.foo() which is defined in your 
manager class. This allows a much better separation of logic and models.

 
> Should there be a better and standardized way to organize huge
> projects in django?

Yes. Making multiple applications. I highly recommend watching this DjangoCon 
video by James Bennett: http://www.youtube.com/watch?v=A-S0tqpPga4

It explains how apps can be made reusable. For example a "blog" application is 
actually not 1 monolythic application, but 8+ tiny apps. User registration and 
account management can also be written as 4+ little apps.

SInce watching that video, my projects are composed by default of 2 apps. A 
"projectname" for the core backend stuff, and a "projectname-site" with the 
templates, settings, and frontend media. This is imho a nicer base to start 
"hooking in" more applications.

More functionality (tagging, rating, accounts, etc.. will all become separate 
apps) which will be hooked in the "projectname-site" project. In case you 
wonder, http://www.djangopackages.com/ gives you quite a wealth of small 
reusable parts that can be hooked in directly.

Greetings,

Diederik

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-us...@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.