Feeling the pain as we speak... trying to figure out how to install SciPy in a virtualenv on a system that I don't have root privileges on (and therefore can't use a package manager rather than compile from source some significant dependencies). Funnily and thankfully though, this group, Continuum Analytics <https://store.continuum.io/cshop/anaconda/>, was mentioned the other day in the Training thread, and just moments ago I happened to be looped back to their site while the desperation of not making much progress was setting in. Turns out they make a Python distro called Anaconda? Looks very cool and pretty tricked out for number crunching and free to ALL and without any root requirement. I haven't tried it out but it looks like it could be a huge help to many.
In the past, at times, I used the Enthought Python distro but got nervous that I was being too coddled by it (and b/c it's only free to academics). Nowadays I mostly go the DIY route (using virtualenv/pip as much as possible). But boy does it get dicey when you don't have root privileges (on Linux and therefore can't use a package manager), or you're on Windows (especially when trying to do the next task--see next), or you're trying to get a package with a lot of dependencies installed into a virtualenv (or non-standard place). Getting a little off topic now... Lately I've been experimenting with Ubuntu and trying to take a Windows diet (figuring out 8 was not my first choice). But it's damn hard to take the diet too far b/c neither system is perfect for all tasks (and lack of iTunes and Picasa in Linux was a big shock to my routine). But despite some inconvenience in my attempt to figure out how to optimally use multiple OSs for work and play, it's been good to find Git for Windows (msysGit) and delve into Pandas (which precipitated wanting to get SciPy installed). What's not been good is a lack of closure on figuring out why I can no longer paste from Windows into an X window. Realizing that I can't use Ctrl+C in ipython on Windows has been another pain. Out of curiosity, do any of you simultaneously use a Windows and Linux machine (without or without Wine, etc.)? Cheers, Matt On Fri, Jul 19, 2013 at 1:54 PM, Nimret Sandhu <[email protected]> wrote: > folks using linux care to chime in on whether they have similar issues > when installing dependencies in python or not? I tend to stay away from > Windows and have never used OS X (well, until the last few weeks at my > current gig). I find OS X to be a more classy environment that Windows but > in many ways even more restrictive. I have really never had any issues with > 3rd party library dependency management but then again I don't do > scientific computing. I generally use jython.org for most of what I do > anyways. > > my points regarding apt-get, maven, etc were just that other > languages/environments have found solutions which work fine for managing > transitive dependencies in general. I didn't mean to imply that you can use > them for managing dependencies in python or the native libraries that > python leverages. > > in java land we generally just create a maven (or ivy or gradle - the new > coolness) config file and declare our immediate library dependencies in > there. The infrastructure then automatically determines the correct > transitive dependencies and downloads the correct versions of all the > necessary libraries. maven has other warts (xml config for one thing) but > the transitive dependencies part generally works rather well. And yes, > folks in the community have gone through the effort of setting up and > maintaining the infrastructure around all of this to ensure that it works > well and is freely available. > > cheers, > - > Nimret Sandhu > http://www.nimret.org > > > On Fri, Jul 19, 2013 at 12:04 PM, James Thiele <[email protected]>wrote: > >> You are totally spot on in your description of how hard it can be to >> install stuff. Years ago the first time I tried to install some Python 2.x >> version from source on OS X I ran into the problem that the filesystem is >> case insensitive. I've tried to install PIL at least 3 times and never got >> the dependecies right. >> >> >> On Fri, Jul 19, 2013 at 11:36 AM, Chris Calloway <[email protected]> wrote: >> >>> On 7/18/2013 9:24 PM, Nimret Sandhu wrote: >>> >>>> looking past the hubris in your email, >>>> >>> >>> Wow. >>> >>> >>> you're talking about dependency >>>> management [DM] >>>> (http://en.wikipedia.org/wiki/**Dependency_hell<http://en.wikipedia.org/wiki/Dependency_hell>) >>>> which >>>> exists in libraries, OSes and pretty much anywhere there is coupling >>>> between reusable code. It can be solved statically by linking against >>>> specific versions which work together but is a little more challenging >>>> to solve dynamically. There are tools in python to manage this though - >>>> http://www.pip-installer.org/ for compile time and >>>> http://www.virtualenv.org/ for runtime. >>>> >>>> java has maven, gradle, etc for compile time DM and tech like osgi for >>>> runtime DM. apt-get on *nix, ports on bsd are tools on OSes for >>>> addressing the same issues. They all work rather painlessly without >>>> 'ridiculous amounts of work'. >>>> >>> >>> This is well beyond dependency hell. It's compiler and linker options >>> and many other things. If you think pip solves these problems, then you >>> haven't tried to use pip to install the dependencies for ipython notebook >>> on Windows. And certainly haven't tried is across the wide variety of >>> Windows and Pythons brought into a classroom. These problems don't even >>> start to get addressed until the setup.py, and then only for the Python >>> bits. >>> >>> If it wasn't a ridiculous amount of work, there wouldn't be a need for >>> prepackaged "sumo" distributions like Canopy or Anaconda. Comparing this to >>> apt-get and ports misses the point. With apt-get, you are sometimes able to >>> install some pieces of the Python scientific stack for the particular >>> system python that apt-get repo for that particular OS and OS version is >>> also managing. But that's rarely what anyone needs. It only works because >>> someone has already done the ridiculous amounts of work packaging binaries >>> for a particular narrow use case. pip gets from a generalized repo of >>> python source with setups that often run compilers and linkers in the case >>> of scientific packages. >>> >>> Virtualenv simply gives you a sandbox to play in to isolate at least the >>> Python parts of the ridiculous amount of work. The ironic thing about >>> pointing at virtualenv.org is that the first three points on the front >>> page under the installation section are version dependency warning. >>> >>> see http://goo.gl/IzI1i >>>> >>> >>> The ironic thing about that link (a Google search for python pip >>> virtualenv scientific computing) is that the top two links are about the >>> problems pip and virtualenv have trying to install the Python scientific >>> stack. >>> >>> The first sentence in the first link was, "For whatever reason, it can >>> be a big pain to get python set up correctly on Mac OS X." Later on it goes >>> on to say, "Next is the tedious process of downloading and installing >>> packages. Unfortunately some of them do not play well with pip and need to >>> be built from the latest source code explicitly. Others simply take a long >>> time to build and you may want to avoid repeating the build when creating a >>> new python virtual environment. Still others don’t build well and need to >>> be installed using pip. Good times." >>> >>> The second link goes on about how, "Now, we would like to install >>> matplotlib, which is a 2-D plotting library for Python. Now Matplotlib is >>> not very friendly with pip/easy_install on many systems. So, we will >>> install from source." >>> >>> >>> -- >>> Sincerely, >>> >>> Chris Calloway http://nccoos.org/Members/cbc >>> office: 3313 Venable Hall phone: (919) 599-3530 >>> mail: Campus Box #3300, UNC-CH, Chapel Hill, NC 27599 >>> >> >> >> >> -- >> Some radio waves were modulated in the creation of this email. >> > >
