Multiple python versions, one dev environment???
Hello! The condensed version of the question would probably be: How does one deal with multiple interpreters and one package where you want to try some changes? The background: I made a trivial change to some package (docutils) to scratch a personal itch, and I want to offer this back to the community (whether it will be accepted or not). Because of this, I ran into some issues. Some facts: 1. Python3 is my main interpreter. 2. The tests of docutils only run under python2. 3. I desire not to touch my distribution's version of site-packagesX-Y. 4. I prefer to have one and only one development directory of my target package. My confusions: 1. Is it possible to have one source control managed directory of some package, which is used by two versions of python? 2. I assume that the *.pyc-files generated while using some python source are version dependent. What is the recommended way to have 'em both installed? 3. The way I have stumped a wall over here, is the puzzle of how to make python2 have a different $PYTHONPATH as python3. I hope to hear how this strategy is silly :) 4. I have contemplated the way of linking the source files from my development directory into user specified site-packages directories. Problem 3. still is valid. 5. Should venv and friends/foes com into play? If so: How? Appreciate any light shed on these issues. Thanks! Joep -- https://mail.python.org/mailman/listinfo/python-list
Re: Multiple python versions, one dev environment???
Hello Javier!
Thanks, those links are helping a bit. And: yes, I am using Archlinux.
But still those links assume that there are totally separate
site-packages* directories installed for both. I am not sure how I
would surpass this distinction between py-X.P and py-Y.Q.
Should I really create a massive amount of symlinks like this?:
| #!/usr/bin/zsh
| for d in ~/src/mypackage/**/*(/); do
|# matches all directories
|mkdir -p "~/src/py-2.7/mypackage/${d#*src/mypackage}"
|mkdir -p "~/src/py-3.4/mypackage/${d#*src/mypackage}"
| done
| for f in ~/src/mypackage/**/^*.pyc(.); do
|# matches all files except for *.pyc
|ln -s "$f" "~/src/py-2.7/mypackage${f#*src/mypackage}"
|ln -s "$f" "~/src/py-3.4/mypackage${f#*src/mypackage}"
| done
...and then set $PYTHONPATH according to the target version in a
#!/usr/local/bin-script?
I can work with this (have not tried though), but there must be a
more elegant solution than symlinking my way forward...
Cheers!
Joep
On Thu, 17 Jul 2014 16:05:27 + (UTC)
Javier wrote:
> Are you using arch linux.
>
>
> I deal with multiple interpreters putting fake executables in
> /usr/local/bin for everything: (python, sphinx, virtualenv, pydoc,
> idle, python-config...) selecting 2 or 3. You can do the same for
> selecting 2.3, 2.5, 2.7. What the scripts do is to detect whether
> it is a system script whose prefix starts with /usr/bin, or whether
> it is a user script. Being in /usr/local/bin they will override
> executables in /usr/bin. Remember to chmod a+x the files
> in /usr/local/bin
>
> http://sindhus.bitbucket.org/manage-python-2-3.html
> http://stackoverflow.com/questions/15400985/how-to-completely-replace-python-3-with-python-2-in-arch-linux
> https://wiki.archlinux.org/index.php/Python#Dealing_with_version_problem_in_build_scripts
>
> I use these scripts, but there is more than one way to do it
>
> /usr/local/bin/python===
> #!/bin/bash
> script=`readlink -f -- "$1"`
> case "$script" in
> /usr/bin*)
> exec python3 "$@"
> ;;
> esac
> exec python2 "$@"
>
> /usr/local/bin/virtualenv=======
> #!/bin/bash
> script=`readlink -f -- "$1"`
> case "$script" in
> /usr/bin*)
> exec virtualenv3 "$@"
> ;;
> esac
>
> exec virtualenv2 "$@"
>
>
>
>
>
>
>
>
> Joep van Delft wrote:
> > Hello!
> >
> > The condensed version of the question would probably be: How does
> > one deal with multiple interpreters and one package where you
> > want to try some changes?
> >
> > The background:
> > I made a trivial change to some package (docutils) to scratch a
> > personal itch, and I want to offer this back to the community
> > (whether it will be accepted or not). Because of this, I ran into
> > some issues.
> >
> > Some facts:
> > 1. Python3 is my main interpreter.
> > 2. The tests of docutils only run under python2.
> > 3. I desire not to touch my distribution's version of
> > site-packagesX-Y.
> > 4. I prefer to have one and only one development directory of
> > my target package.
> >
> > My confusions:
> > 1. Is it possible to have one source control managed directory of
> > some package, which is used by two versions of python?
> > 2. I assume that the *.pyc-files generated while using some python
> > source are version dependent. What is the recommended way to
> > have 'em both installed?
> > 3. The way I have stumped a wall over here, is the puzzle of how
> > to make python2 have a different $PYTHONPATH as python3. I hope to
> > hear how this strategy is silly :)
> > 4. I have contemplated the way of linking the source files from my
> > development directory into user specified site-packages
> > directories. Problem 3. still is valid.
> > 5. Should venv and friends/foes com into play? If so: How?
> >
> > Appreciate any light shed on these issues.
> >
> > Thanks!
> >
> >
> >Joep
> >
> >
> >
--
https://mail.python.org/mailman/listinfo/python-list
Re: Multiple python versions, one dev environment???
On Thu, 17 Jul 2014 15:41:44 -0400 Ned Batchelder wrote: > > For testing one project on multiple versions of Python, use tox. > Its entire reason for being is to test Python code against multiple > environments, generally for different Python versions, but possibly > for other reasons, like different versions of dependencies. > > Tox will manage the virtualenvs for you, it makes multi-version > testing very simple. > Excellent, Ned and Akira, I will look into it! Cheers, Joep -- https://mail.python.org/mailman/listinfo/python-list
strace of python shows nonsense
Hi there, I am puzzled at how I borked my installation. Python loads slow on my machine, and I decided to use strace and /usr/bin/time to see what is actually happening. # sync && echo 3 > /proc/sys/vm/drop_caches $ /usr/bin/time python2 -c "" 0.19user 0.04system 0:01.22elapsed 19%CPU (0avgtext+0avgdata 4200maxresident)k 7312inputs+0outputs (4major+1145minor)pagefaults 0swaps And when all is in memory: $ /usr/bin/time python2 -c "" 0.19user 0.01system 0:00.21elapsed 98%CPU (0avgtext+0avgdata 4216maxresident)k 0inputs+0outputs (0major+1153minor)pagefaults 0swaps $ /usr/bin/time python2 -c "import argparse" 0.36user 0.02system 0:00.39elapsed 98%CPU (0avgtext+0avgdata 5752maxresident)k 0inputs+0outputs (0major+1699minor)pagefaults 0swaps .2 and .4 seconds to not even get started when all disk I/O is cached. So what is happening here? $ strace -c python2 -c "" % time seconds usecs/call callserrors syscall -- --- --- - - 93.260.001910 8 230 168 open 3.660.75 9 8 mprotect 3.080.63 197 fstat64 0.000.00 0 172 read 0.000.00 063 close <...> $ strace -c python2 -c "import argparse" % time seconds usecs/call callserrors syscall -- --- --- - - 51.890.003732 13 290 read 47.290.003401 18 192 155 stat64 0.820.59 0 129 mmap2 0.000.00 0 549 443 open 0.000.00 0 107 close <...> What puzzles me, is the amount of errors for open and stat64. There are references to stuff I don't know (~/GNUStep directory? Stuff under site-packages that does not exist? Subdirs of site-packages that are not included in sys.path?) What is python doing there, and why? And, more importantly, how can this be corrected? Probably irrelevant, but Python2 version 2.7.3, Archlinux (current as of previous weekend). Thanks, Joep -- http://mail.python.org/mailman/listinfo/python-list
