Re: Incorrect Python Version Being Used

[Fred Stluka]

Tom,

Excellent article!  Thanks!

--Fred

Fred Stluka -- mailto:f...@bristle.com -- http://bristle.com/~fred/
Bristle Software, Inc -- http://bristle.com -- Glad to be of service!
Open Source: Without walls and fences, we need no Windows or Gates.


On 11/15/12 12:46 PM, Tom Evans wrote:

On Thu, Nov 15, 2012 at 3:49 PM, rh  wrote:

In the djangocon lightning talk there was mention of this disconnect
between "works on runserver" but not in production env.

Not a problem restricted to django. I've been checking out a lot of
frameworks in different languages and most suffer the same malady.
i.e. everything works great locally but hauling it all into a production
env. becomes a project in itself.

It's hard to document because each env. will be different.


This is an interesting problem, which you can work around to a certain
extent. Making predictable and reproducible deployments of code is an
integral part of Software Configuration Management. SCM is a big
topic, so I'm not going to go into crazy details, but here are some
details of how we mitigate these risks at $JOB

1) Use virtualenv and pip

Virtualenv is essential. It allows you to separate the libraries you
use in your deployment from the any other python package. I recommend
using "--no-site-packages" to force virtualenv to ignore any system
site packages, but sometimes it can be necessary.

Pip is the other side of this. Pip is a simple way to specify python
packages to install. You can explicitly reference versions of
packages, or ranges of packages. Eg, "Django>1.4,<1.5" specifies that
the most recent release in the 1.4 cycle is correct. You can also have
a single file that lists all the packages required for your project to
run, usually called 'requirements.txt'. Pip will take that file, and
install all listed packages along with any dependencies.


2) Your project structure is code, and should live in source control.

Your project structure is all the files and folders that make up a
deployment, but aren't actual python code. Eg, if your deployment for
django project 'myproj' looks like this:

.
├── htdocs/
├── logs/
├── project/
│   ├── manage.py
│   └── myproj/
└── scripts/

The 'myproj' directory contains the project source code and is usually
version controlled. However, everything listed there is part of the
project deployment, and changes there must be tracked. If your project
requires a 'logs' directory, then a logs directory must be a versioned
resource that you can check out at any point in it's past.

Just as importantly, any changes that you make in the project
structure in development must be replicated to the production
deployments, and therefore must be tracked.


3) Django apps are also python libraries.

A 'django app', regardless of what it does, is also a python library.
Therefore, make sure you give it a setup.py script using distutils
that will allow you to do standard python library packaging operations
on it. Pip will happily install an app from source control, it will
even install in a manner that allows you to edit and commit changes
back to source control.
This allows you to install packages in development in the same way as
production.


4) Write a bootstrap script.

A bootstrap script is a simple shell script that does two things -
sets up a virtualenv environment, and installs a set of packages.
This combination of storing project structure in source control, using
virtualenv and pip, having a bootstrap script and having a
requirements.txt listing your required packages allows you to simply
checkout a project and run it's bootstrap script to end up with an
identical, repeatable project environment.


6) Use the capabilities of your source control to assemble a deployment

This is the tricky one where I tend to lose people! We use subversion
for source control. Subversion allows us to specify 'externals', which
are secondary subversion repositories that we want to be automatically
checked out in specific locations within another repository. You can
also have 'file externals', which is a single file (rather than an
entire repository tree) that is checked out at a specific location.

The benefit of this is that you can use your project structure
repository as a basis for a deployment on a single site, by
duplicating it for each server that you wish to deploy on.
On each one, you would have an external adding the project source code
in the right place. You would have a file external that pulls in a
'requirements.txt' file correct for this project into the right place.
You would have another file external that pulls in the correct
settings.py (or settings_local.py if that is what you prefer).


7) Use South

Use south for database migrations. If you aren't doing this already,
you should be! Schemas rarely stay static, you need good ways of
managing that change.


8) Use Fabric

Re: Incorrect Python Version Being Used

[Tom Evans]

On Thu, Nov 15, 2012 at 3:49 PM, rh  wrote:
> In the djangocon lightning talk there was mention of this disconnect
> between "works on runserver" but not in production env.
>
> Not a problem restricted to django. I've been checking out a lot of
> frameworks in different languages and most suffer the same malady.
> i.e. everything works great locally but hauling it all into a production
> env. becomes a project in itself.
>
> It's hard to document because each env. will be different.
>

This is an interesting problem, which you can work around to a certain
extent. Making predictable and reproducible deployments of code is an
integral part of Software Configuration Management. SCM is a big
topic, so I'm not going to go into crazy details, but here are some
details of how we mitigate these risks at $JOB

1) Use virtualenv and pip

Virtualenv is essential. It allows you to separate the libraries you
use in your deployment from the any other python package. I recommend
using "--no-site-packages" to force virtualenv to ignore any system
site packages, but sometimes it can be necessary.

Pip is the other side of this. Pip is a simple way to specify python
packages to install. You can explicitly reference versions of
packages, or ranges of packages. Eg, "Django>1.4,<1.5" specifies that
the most recent release in the 1.4 cycle is correct. You can also have
a single file that lists all the packages required for your project to
run, usually called 'requirements.txt'. Pip will take that file, and
install all listed packages along with any dependencies.


2) Your project structure is code, and should live in source control.

Your project structure is all the files and folders that make up a
deployment, but aren't actual python code. Eg, if your deployment for
django project 'myproj' looks like this:

.
├── htdocs/
├── logs/
├── project/
│   ├── manage.py
│   └── myproj/
└── scripts/

The 'myproj' directory contains the project source code and is usually
version controlled. However, everything listed there is part of the
project deployment, and changes there must be tracked. If your project
requires a 'logs' directory, then a logs directory must be a versioned
resource that you can check out at any point in it's past.

Just as importantly, any changes that you make in the project
structure in development must be replicated to the production
deployments, and therefore must be tracked.


3) Django apps are also python libraries.

A 'django app', regardless of what it does, is also a python library.
Therefore, make sure you give it a setup.py script using distutils
that will allow you to do standard python library packaging operations
on it. Pip will happily install an app from source control, it will
even install in a manner that allows you to edit and commit changes
back to source control.
This allows you to install packages in development in the same way as
production.


4) Write a bootstrap script.

A bootstrap script is a simple shell script that does two things -
sets up a virtualenv environment, and installs a set of packages.
This combination of storing project structure in source control, using
virtualenv and pip, having a bootstrap script and having a
requirements.txt listing your required packages allows you to simply
checkout a project and run it's bootstrap script to end up with an
identical, repeatable project environment.


6) Use the capabilities of your source control to assemble a deployment

This is the tricky one where I tend to lose people! We use subversion
for source control. Subversion allows us to specify 'externals', which
are secondary subversion repositories that we want to be automatically
checked out in specific locations within another repository. You can
also have 'file externals', which is a single file (rather than an
entire repository tree) that is checked out at a specific location.

The benefit of this is that you can use your project structure
repository as a basis for a deployment on a single site, by
duplicating it for each server that you wish to deploy on.
On each one, you would have an external adding the project source code
in the right place. You would have a file external that pulls in a
'requirements.txt' file correct for this project into the right place.
You would have another file external that pulls in the correct
settings.py (or settings_local.py if that is what you prefer).


7) Use South

Use south for database migrations. If you aren't doing this already,
you should be! Schemas rarely stay static, you need good ways of
managing that change.


8) Use Fabric

Fabric is a clever system for writing scripts to do things on remote
servers. Work out the steps required to update your project source
code, libraries, deploy migrations, and codify it into a fabric
fabfile. Never make a mistake in deployment again!


There is more to it than this of course (there is a lot more to each
of these steps even!), but these simple ideas allowed us reliable and
predictable deployments and

Re: Incorrect Python Version Being Used

[Bestrafung]

Thanks for the info. I built mine using:
./configure --with-apxs=/usr/local/apache/bin/apxs 
--with-python=/opt/python2.7/bin/python

I'm fairly certain everything is good there. I'm able to load both projects 
individually using "python manage.py runserver 0.0.0.0:8000" without issue. 
The trouble now comes from trying to access them via the Apache server. 
There's some kind of problem with the bootstrap .wsgi file(s) and I also 
need to figure out how to load both at the same time as Apache spits an 
error and refuses to build the vhost with multiple projects. I hope some of 
this helps others but my issue is definitely in the mod_wsgi/Apache setup 
now.


On Tuesday, November 6, 2012 1:45:49 PM UTC-5, Nikolas Stevenson-Molnar 
wrote:
>
>  If you're running via mod_wsgi, then you need to look at which version of 
> Python mod_wsgi was built with. Probably *not *2.7 in that case.
>
>
> http://code.google.com/p/modwsgi/wiki/InstallationIssues#Multiple_Python_Versions
>
> _Nik
>  

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/prxw6dH87zsJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Incorrect Python Version Being Used

[Nikolas Stevenson-Molnar]

If you're running via mod_wsgi, then you need to look at which version
of Python mod_wsgi was built with. Probably /not /2.7 in that case.

http://code.google.com/p/modwsgi/wiki/InstallationIssues#Multiple_Python_Versions

_Nik

On 11/6/2012 10:02 AM, Bestrafung wrote:
> Thank you for the info. That's how I've been doing it but assumed
> something wasn't right as it wasn't working and I keep getting errors.
> I think I've ruled out python as the problem Need to start looking at
> the mod_wsgi and Apache setup.
>
> I have another post regarding Apache errors I will get back to, is
> there a way to close this topic?
>
> On Tuesday, November 6, 2012 11:14:53 AM UTC-5, Nikolas
> Stevenson-Molnar wrote:
>
> Whenever you run a Django command (e.g., startapp, runserver),
> just use
> the full path to your Python 2.7 interpreter. For example:
>
> $ /opt/python2.7/bin/python manage.py runserver 80
>
> Similarly, if you're installing packages via pip or easy_install:
>
> $ /opt/python2.7/bin/pip install some_package
> $ /opt/python2.7/bin/easy_install some_package
>
> _Nik
>
> On 11/6/2012 7:09 AM, Bestrafung wrote:
> > I have been running into this problem for a long while trying to
> setup
> > my first Django project and I keep coming back to this problem.
> I am
> > relatively new when it come to Linux, I'm learning but still have a
> > long way to go. I am using CentOS 5.8 cPanel which comes with
> Python
> > 2.4. Following instructions I found online for setting up Django
> with
> > cPanel I compiled/installed Python 2.7 from source to
> /opt/python2.7
> > and set an alias (alias python="/opt/python2.7/bin/python") in the
> > root and user's .bash_profile and python 2.7 launches as
> expected when
> > you use the python command. Every time I try to start a project or
> > work with it however the system default python 2.4 keeps popping
> up. I
> > think this may be the single cause of all of my issues. When
> starting
> > a project I have just appended the python command to make sure it
> > started with the correct version. I believe that once the
> project is
> > started it isn't working because everything is still trying to use
> > 2.4. Is there a way to make Django use my 2.7 install instead of
> 2.4
> > without breaking the system by making the 2.7 system default?
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Django users" group.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msg/django-users/-/WqpZIEJqalcJ
> .
> > To post to this group, send email to django...@googlegroups.com
> .
> > To unsubscribe from this group, send email to
> > django-users...@googlegroups.com .
> > For more options, visit this group at
> > http://groups.google.com/group/django-users?hl=en
> .
>
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-users/-/OVUWdCah_fwJ.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Incorrect Python Version Being Used

[Bestrafung]

I'm not familiar with how to do this but the suggestion is noted. I will 
look into it. Thanks.

On Tuesday, November 6, 2012 11:26:41 AM UTC-5, Thomas wrote:
>
> Use virtualenv. Always. 
>
>  - Tom 
> > -- 
> > You received this message because you are subscribed to the Google 
> > Groups "Django users" group. 
> > To view this discussion on the web visit 
> > https://groups.google.com/d/msg/django-users/-/WqpZIEJqalcJ. 
> > To post to this group, send email to 
> > django...@googlegroups.com. 
>
> > To unsubscribe from this group, send email to 
> > django-users...@googlegroups.com . 
> > For more options, visit this group at 
> > http://groups.google.com/group/django-users?hl=en. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/yd9js8Dy6L0J.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Incorrect Python Version Being Used

[Bestrafung]

Thank you for the info. That's how I've been doing it but assumed something 
wasn't right as it wasn't working and I keep getting errors. I think I've 
ruled out python as the problem Need to start looking at the mod_wsgi and 
Apache setup.

I have another post regarding Apache errors I will get back to, is there a 
way to close this topic?

On Tuesday, November 6, 2012 11:14:53 AM UTC-5, Nikolas Stevenson-Molnar 
wrote:
>
> Whenever you run a Django command (e.g., startapp, runserver), just use 
> the full path to your Python 2.7 interpreter. For example: 
>
> $ /opt/python2.7/bin/python manage.py runserver 80 
>
> Similarly, if you're installing packages via pip or easy_install: 
>
> $ /opt/python2.7/bin/pip install some_package 
> $ /opt/python2.7/bin/easy_install some_package 
>
> _Nik 
>
> On 11/6/2012 7:09 AM, Bestrafung wrote: 
> > I have been running into this problem for a long while trying to setup 
> > my first Django project and I keep coming back to this problem. I am 
> > relatively new when it come to Linux, I'm learning but still have a 
> > long way to go. I am using CentOS 5.8 cPanel which comes with Python 
> > 2.4. Following instructions I found online for setting up Django with 
> > cPanel I compiled/installed Python 2.7 from source to /opt/python2.7 
> > and set an alias (alias python="/opt/python2.7/bin/python") in the 
> > root and user's .bash_profile and python 2.7 launches as expected when 
> > you use the python command. Every time I try to start a project or 
> > work with it however the system default python 2.4 keeps popping up. I 
> > think this may be the single cause of all of my issues. When starting 
> > a project I have just appended the python command to make sure it 
> > started with the correct version. I believe that once the project is 
> > started it isn't working because everything is still trying to use 
> > 2.4. Is there a way to make Django use my 2.7 install instead of 2.4 
> > without breaking the system by making the 2.7 system default? 
> > -- 
> > You received this message because you are subscribed to the Google 
> > Groups "Django users" group. 
> > To view this discussion on the web visit 
> > https://groups.google.com/d/msg/django-users/-/WqpZIEJqalcJ. 
> > To post to this group, send email to 
> > django...@googlegroups.com. 
>
> > To unsubscribe from this group, send email to 
> > django-users...@googlegroups.com . 
> > For more options, visit this group at 
> > http://groups.google.com/group/django-users?hl=en. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/OVUWdCah_fwJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Incorrect Python Version Being Used

[Thomas Lockhart]

On 11/6/12 7:09 AM, Bestrafung wrote:
I have been running into this problem for a long while trying to setup 
my first Django project and I keep coming back to this problem. I am 
relatively new when it come to Linux, I'm learning but still have a 
long way to go. I am using CentOS 5.8 cPanel which comes with Python 
2.4. Following instructions I found online for setting up Django with 
cPanel I compiled/installed Python 2.7 from source to /opt/python2.7 
and set an alias (alias python="/opt/python2.7/bin/python") in the 
root and user's .bash_profile and python 2.7 launches as expected when 
you use the python command. Every time I try to start a project or 
work with it however the system default python 2.4 keeps popping up. I 
think this may be the single cause of all of my issues. When starting 
a project I have just appended the python command to make sure it 
started with the correct version. I believe that once the project is 
started it isn't working because everything is still trying to use 
2.4. Is there a way to make Django use my 2.7 install instead of 2.4 
without breaking the system by making the 2.7 system default?

Use virtualenv. Always.

- Tom

--
You received this message because you are subscribed to the Google 
Groups "Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/WqpZIEJqalcJ.

To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.


--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Re: Incorrect Python Version Being Used

[Nikolas Stevenson-Molnar]

Whenever you run a Django command (e.g., startapp, runserver), just use
the full path to your Python 2.7 interpreter. For example:

$ /opt/python2.7/bin/python manage.py runserver 80

Similarly, if you're installing packages via pip or easy_install:

$ /opt/python2.7/bin/pip install some_package
$ /opt/python2.7/bin/easy_install some_package

_Nik

On 11/6/2012 7:09 AM, Bestrafung wrote:
> I have been running into this problem for a long while trying to setup
> my first Django project and I keep coming back to this problem. I am
> relatively new when it come to Linux, I'm learning but still have a
> long way to go. I am using CentOS 5.8 cPanel which comes with Python
> 2.4. Following instructions I found online for setting up Django with
> cPanel I compiled/installed Python 2.7 from source to /opt/python2.7
> and set an alias (alias python="/opt/python2.7/bin/python") in the
> root and user's .bash_profile and python 2.7 launches as expected when
> you use the python command. Every time I try to start a project or
> work with it however the system default python 2.4 keeps popping up. I
> think this may be the single cause of all of my issues. When starting
> a project I have just appended the python command to make sure it
> started with the correct version. I believe that once the project is
> started it isn't working because everything is still trying to use
> 2.4. Is there a way to make Django use my 2.7 install instead of 2.4
> without breaking the system by making the 2.7 system default?
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-users/-/WqpZIEJqalcJ.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.

Incorrect Python Version Being Used

[Bestrafung]

I have been running into this problem for a long while trying to setup my 
first Django project and I keep coming back to this problem. I am 
relatively new when it come to Linux, I'm learning but still have a long 
way to go. I am using CentOS 5.8 cPanel which comes with Python 2.4. 
Following instructions I found online for setting up Django with cPanel I 
compiled/installed Python 2.7 from source to /opt/python2.7 and set an 
alias (alias python="/opt/python2.7/bin/python") in the root and user's 
.bash_profile and python 2.7 launches as expected when you use the python 
command. Every time I try to start a project or work with it however the 
system default python 2.4 keeps popping up. I think this may be the single 
cause of all of my issues. When starting a project I have just appended the 
python command to make sure it started with the correct version. I believe 
that once the project is started it isn't working because everything is 
still trying to use 2.4. Is there a way to make Django use my 2.7 install 
instead of 2.4 without breaking the system by making the 2.7 system default?

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-users/-/WqpZIEJqalcJ.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en.