good morning, developers! TL;DR:
i will be installing anaconda and setting it in the system PATH so that your python will default to 2.7, as well as it taking over management of all of the sci-py packages. this is potentially a big change, so i'll be testing locally on my staging instance before deployment to the wide world. deployment is *tentatively* next monday, march 2nd. a little background: the jenkins test infra is currently (and happily) managed by a set of tools that allow me to set up and deploy new workers, manage their packages and make sure that all spark and research projects can happily and successfully build. we're currently at the state where ~50 or so packages are installed and configured on each worker. this is getting a little cumbersome, as the package-to-build dep tree is getting pretty large. the biggest offender is the science-based python infrastructure. everything is blindly installed w/yum and pip, so it's hard to control *exactly* what version of any given library is as compared to what's on a dev's laptop. the solution: anaconda (https://store.continuum.io/cshop/anaconda/)! everything is centralized! i can manage specific versions much easier! what this means to you: * python 2.7 will be the default system python. * 2.6 will still be installed and available (/usr/bin/python or /usr/bin/python/2.6) what you need to do: * install anaconda, have it update your PATH * build locally and try to fix any bugs (for spark, this "should just work") * if you have problems, reach out to me and i'll see what i can do to help. if we can't get your stuff running under python2.7, we can default to 2.6 via a job config change. what i will be doing: * setting up anaconda on my staging instance and spot-testing a lot of builds before deployment please let me know if there are any issues/concerns... i'll be posting updates this week and will let everyone know if there are any changes to the Plan[tm]. your friendly devops engineer, shane
