We use pip to install and manage all 3rd party libraries. To do this, we create a distutils.cfg on the local workstation of each developer who will be using pip to control the directory layout:
chad$ cat /usr/local/python-2.7.4/lib/python2.7/distutils/distutils.cfg [install] install-base=$PYTHON_THIRD_PARTY install-purelib=$PYTHON_THIRD_PARTY/lib/pure install-platlib=$PYTHON_THIRD_PARTY/lib/$OS_ARCH install-scripts=$PYTHON_THIRD_PARTY/bin install-headers=$PYTHON_THIRD_PARTY/include install-data=$PYTHON_THIRD_PARTY/data then we add $PYTHON_THIRD_PARTY/lib/pure and $PYTHON_THIRD_PARTY/lib/$OS_ARCH to the PYTHONPATH and $PYTHON_THIRD_PARTY/bin to the PATH. note that the $PYTHON_THIRD_PARTY variable includes a version directory (e.g. 2.6, 2.7) which keeps versions completely independent of each other. This is particularly important for executable scripts (in the bin dir), because the path to the interpreter gets baked into the script during install. when using the default layout, all versions of python share the same bin directory, so if you installed version 1.1 of packageX using python 2.6, then 1.2 of packageX using python 2.7, the executable for packageX in the bin directory will use version 1.2 with python 2.7. we need more control than that, hence the distutils.cfg file. Also note that this layout works for us because our python executables are in the same location on osx and linux, and so the interpreter specified on the first line of the scripts (e.g. #!/usr/local/python-2.7.4/bin/python) will exist regardless of which OS the package was installed from. A safer solution would be to set install-scripts=$PYTHON_THIRD_PARTY/$OS_ARCH/bin Lastly, we use a package management system to set our environment variables. we have a custom in-house solution that we use currently, but I've been collaborating with a few individuals from different studios on a next-generation open source solution incorporating the best ideas from different studios: https://github.com/nerdvegas/rez. version 2.0 (on the 2.0 branch) is going to be a massive update and a breaking change from 1.0, so you might want to wait a month or so before adopting, but feel free to jump in and let us know what you think. chad. On Mon, May 12, 2014 at 9:52 AM, Tony Barbieri <great...@gmail.com> wrote: > We use a similar technique as Erkan, but we don't rely on putting all of > our paths in PYTHONPATH any longer. Instead, the only path in PYTHONPATH > is the location of a sitecustomize.py file. sitecustomize.py is a magic > file that python will import automatically at initialization time. One > trick with it is you can only ever have one. When python scans it's > initial lib paths, if it finds a file named sitecustomize.py it will import > and execute it. > > The issue that cropped up from using PYTHONPATH is it will insert all > paths found in it to the beginning of the sys.path. If it's only a single > path, then using PYTHONPATH should be fine. If you're going to have > multiple network locations in it, you may want to look at alternatives due > to paths found in PYTHONPATH being inserted before the standard python lib > paths. Because the network locations were at the beginning, python would > stat the network files system for the standard python modules like os, sys, > etc causing unnecessary slowdowns. > > Instead what we did is we created a second environment variable, > PSYOP_PYTHONPATH, and have logic in our sitecustomize.py file to take the > paths present in PSYOP_PYTHONPATH and append them to the sys.path at python > startup. That's the simple version of what we are doing with > sitecustomize.py, we also have a bunch of additional code for dealing with > project context, per show code, etc. > > Of course what Erkan said is correct, I just wanted to point out an > alternative if you notice some slowdowns once you've implement it. > > Best, > > > > On Mon, May 12, 2014 at 12:32 PM, Erkan Özgür Yılmaz > <eoyil...@gmail.com>wrote: > >> Hi, >> >> I should repeat my reply also here, >> >> I try to put everything to the server. >> >> I place Stalker, Anima, SQLAlchemy (has a compiled query engine --I >> think-- but I don't care about the Windows workstations are not being able >> use it, it is already fast), Jinja2, comtypes and other python libraries to >> server, setup PYTHONPATH to include those paths. >> >> And install PySide, PyQt4, psycopg2 to workstations to all the >> applications that has an internal Python interpretter (Maya, Nuke, Houdini >> etc.) plus to the stand alone interpreter (python 2.7). >> >> >> E.Ozgur Yilmaz >> eoyilmaz.blogspot.com >> >> >> On Mon, May 12, 2014 at 6:20 PM, Fredrik Averpil < >> fredrik.aver...@gmail.com> wrote: >> >>> Hi, >>> >>> I've been meaning to learn how to efficiently serve a python module to >>> all workstations/render blades without actually installing it locally... >>> >>> So far I've just copied already compiled modules from the site-packages >>> folder from a local installation and then I keep these in folders such as >>> python26_win7_site-packages, python26_linux_site-packages on the server. >>> Not the cleanest and nicest way of dealing with this, I guess. But what's >>> nice is my scripts just need to do something like this: >>> >>> sys.path.append('/server/share/modules/') >>> import modulex >>> >>> So it's easy to make new modules accessible for machines quickly. >>> >>> My question is; how would you guys deal with this when building from >>> source? >>> Eggs in separate OS folders? >>> Is it at all possible to build eggs for Win/Linux/OSX from one and the >>> same operating system? >>> I'm usually on Windows (7). Should I rather be using a Linux environment >>> for this type of work? >>> >>> Is there an RTFM to this (and/or building eggs)? >>> Please point me towards any literature or write-up worth reading as I'm >>> new to making eggs (and building from source for that matter). >>> >>> To be less abstract, I'm looking to build >>> stalker<https://pypi.python.org/pypi/stalker>from source and have it >>> available (although stored on the server) for all >>> machines (Win/Linux). >>> >>> Thanks in advance for any pointers. >>> >>> // Fredrik >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Python Programming for Autodesk Maya" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to python_inside_maya+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/python_inside_maya/CAD%3DwhWO%2BtdFE34yhwgbWc2n%2BuFqUb_XRD1oOO%3DyVEDcms%2BKGtg%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAD%3DwhWO%2BtdFE34yhwgbWc2n%2BuFqUb_XRD1oOO%3DyVEDcms%2BKGtg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Python Programming for Autodesk Maya" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to python_inside_maya+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/python_inside_maya/CAGNmyx69utr_3DwbbSPgC6Oo%3DTzjgaS%3DcM8mu%2BhGg4MSYtcpag%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAGNmyx69utr_3DwbbSPgC6Oo%3DTzjgaS%3DcM8mu%2BhGg4MSYtcpag%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > -tony > > -- > You received this message because you are subscribed to the Google Groups > "Python Programming for Autodesk Maya" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to python_inside_maya+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/python_inside_maya/CAJhmvsQOUSf54Qr%2BpHVwcBkyRXt47u_W6NsxcxKM%3DTasyYCoeg%40mail.gmail.com<https://groups.google.com/d/msgid/python_inside_maya/CAJhmvsQOUSf54Qr%2BpHVwcBkyRXt47u_W6NsxcxKM%3DTasyYCoeg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group. To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_maya+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAGq9Q7HeY%3D9xmbBLJ8KQL-ZseEcK9CEx7LhvV6FdVeqCsRpcAA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
