[scl.org] How to submit patches for SCLs
Specifically looking at SCL packages that are originally from RHSCL. I think that means I would need to send patches for the spec files to RedHat but I can't figure out where those specs actually live (i.e. which git repo to send a patch in for). Anyone have a direction to look at? Taking rh-python34 as an example, SCLo links to COPR (https://copr.fedorainfracloud.org/coprs/rhscl/rh-python34-el7) but COPR links to 404'd git repo (http://copr-dist-git.fedorainfracloud.org/cgit/rhscl/rh-python34-el7/rh-python34.git). --Noah signature.asc Description: Message signed with OpenPGP using GPGMail ___ SCLorg mailing list SCLorg@redhat.com https://www.redhat.com/mailman/listinfo/sclorg
Re: [scl.org] Problems with Python scripts that use SCL runtimes
I must concede that scl is not as convenient as just having both python and python3 installed side by side, as they are on Ubuntu and Fedora. I really don't think developer convenience is the differentiator point of CentOS/RHEL however, CentOS/RHEL do well in managed environments. During my start-up experience, our desktops were Fedora, and the product ran on CentOS, depending on the version. That is not a bad way to do it if your governance/systems allows it. I am feeling your pain, however. My current organization runs Python 3 on Windows and deploys on CentOS, as the best solution to some concerns about security and what developers are allowed/encouraged to do. Dan Davis, Systems/Applications Architect (Contractor), Office of Computer and Communications Systems, National Library of Medicine, NIH -Original Message- From: Nick Coghlan [mailto:ncogh...@redhat.com] Sent: Thursday, June 09, 2016 2:41 PM To: Davis, Daniel (NIH/NLM) [C] Cc: sclorg@redhat.com Subject: Re: [scl.org] Problems with Python scripts that use SCL runtimes On Thu, Jun 9, 2016 at 7:59 AM, Davis, Daniel (NIH/NLM) [C] wrote: > Nick, > > We also encountered this issue, and we wished DevOps and developers not to > have the issue. > So, our development users, DevOps account, and CI account all have in their > ~/.bash_profile: > > test -f /opt/rh/rh-python34/enable && source > /opt/rh/rh-python34/enable > > This enable script is different from "scl enable" in that it does not start a > sub-process. > > This could be done for everyone via /etc/profile.d/, but we haven't gone that > far - we would want then to test whether the user is in a group that gets > Python 3, because we don't want this for everyone - vendor provided python > scripts may sadly start with "#!/usr/bin/env python"! > > We also place this line in /etc/sysconfig/httpd, but now that we have > switched from mod_wsgi to Phusion Passenger, it is not clear that this is > needed. > > Does this address your issue? Not quite, although I do like it as a way to improve the situation in managed environments that only want to support a single non-system-Python version at a time. It occurs to me that a potential better illustrator than pipsi of the user experience problem I see with the status quo is the "vex" virtual environment manager, which I use to switch between Python 2 and Python 3 for different projects (my own primary OS is Fedora, so I have both installed as system packages): vex --python `which python2` -m py2-example vex --python `which python3` -m py3-example Given virtual environments created that way, I no longer need to remember which ones are Python 2 and which are Python 3 - "vex py2-example" and "vex py3-example" will automatically activate the right runtime. SCLs don't currently offer that same degree of seamlessness, since they rely on LD_LIBRARY_PATH being set in the environment, rather than building knowledge of the SCLs additional shared library directories directly into the binaries. They certainly *can* be used without it, but it's an ongoing source of friction since Python tools generally assume that a Python runtime will know where to find its shared libraries. Cheers, Nick. -- Nick Coghlan Fedora Environments & Stacks Red Hat Developer Experience, Brisbane Software Development Workflow Designer & Process Architect ___ SCLorg mailing list SCLorg@redhat.com https://www.redhat.com/mailman/listinfo/sclorg
Re: [scl.org] Problems with Python scripts that use SCL runtimes
On Thu, Jun 9, 2016 at 7:59 AM, Davis, Daniel (NIH/NLM) [C] wrote: > Nick, > > We also encountered this issue, and we wished DevOps and developers not to > have the issue. > So, our development users, DevOps account, and CI account all have in their > ~/.bash_profile: > > test -f /opt/rh/rh-python34/enable && source > /opt/rh/rh-python34/enable > > This enable script is different from "scl enable" in that it does not start a > sub-process. > > This could be done for everyone via /etc/profile.d/, but we haven't gone that > far - we would want then to test whether the user is in a group that gets > Python 3, because we don't want this for everyone - vendor provided python > scripts may sadly start with "#!/usr/bin/env python"! > > We also place this line in /etc/sysconfig/httpd, but now that we have > switched from mod_wsgi to Phusion Passenger, it is not clear that this is > needed. > > Does this address your issue? Not quite, although I do like it as a way to improve the situation in managed environments that only want to support a single non-system-Python version at a time. It occurs to me that a potential better illustrator than pipsi of the user experience problem I see with the status quo is the "vex" virtual environment manager, which I use to switch between Python 2 and Python 3 for different projects (my own primary OS is Fedora, so I have both installed as system packages): vex --python `which python2` -m py2-example vex --python `which python3` -m py3-example Given virtual environments created that way, I no longer need to remember which ones are Python 2 and which are Python 3 - "vex py2-example" and "vex py3-example" will automatically activate the right runtime. SCLs don't currently offer that same degree of seamlessness, since they rely on LD_LIBRARY_PATH being set in the environment, rather than building knowledge of the SCLs additional shared library directories directly into the binaries. They certainly *can* be used without it, but it's an ongoing source of friction since Python tools generally assume that a Python runtime will know where to find its shared libraries. Cheers, Nick. -- Nick Coghlan Fedora Environments & Stacks Red Hat Developer Experience, Brisbane Software Development Workflow Designer & Process Architect ___ SCLorg mailing list SCLorg@redhat.com https://www.redhat.com/mailman/listinfo/sclorg
Re: [scl.org] Problems with Python scripts that use SCL runtimes
Nick, We also encountered this issue, and we wished DevOps and developers not to have the issue. So, our development users, DevOps account, and CI account all have in their ~/.bash_profile: test -f /opt/rh/rh-python34/enable && source /opt/rh/rh-python34/enable This enable script is different from "scl enable" in that it does not start a sub-process. This could be done for everyone via /etc/profile.d/, but we haven't gone that far - we would want then to test whether the user is in a group that gets Python 3, because we don't want this for everyone - vendor provided python scripts may sadly start with "#!/usr/bin/env python"! We also place this line in /etc/sysconfig/httpd, but now that we have switched from mod_wsgi to Phusion Passenger, it is not clear that this is needed. Does this address your issue? Dan Davis, Systems/Applications Architect (Contractor), Office of Computer and Communications Systems, National Library of Medicine, NIH -Original Message- From: sclorg-boun...@redhat.com [mailto:sclorg-boun...@redhat.com] On Behalf Of Nick Coghlan Sent: Thursday, June 09, 2016 10:07 AM To: sclorg@redhat.com Subject: [scl.org] Problems with Python scripts that use SCL runtimes Hi folks, The question of the explicit "scl enable" step for using SCLs is one that's come up a few times in different contexts, and I finally sat down to document a relatively straightforward scenario that illustrates the problem: https://github.com/ncoghlan/scl-pipsi-demo#demonstrating-the-problem What the demo covers doing is: 1. Enable the Python 3.5 SCL 2. Use pipsi within that SCL to install the pygmentize script for a particular user within a virtual environment configured to use that particular runtime 3. Disable the SCL 4. Show that the installed script doesn't work due to the runtime being unable to find its shared libraries Ideally, the fact I used an SCL to install (and subsequently run) pygmentize would be an invisible implementation detail. However, at the moment, it's visible, since Python's virtual environment machinery only fixes paths for Python level imports - it doesn't touch the paths for operating system level dynamic library loading. As far as I am aware, changing the SCL binaries to set RPATH and/or RUNPATH should be sufficient to resolve the problem, at least with pipsi managed script installation (plain "--user" installations are likely to still have problems, but I'm more comfortable with a restriction saying not to use --user installs in conjunction with SCLs) Regards, Nick. -- Nick Coghlan Fedora Environments & Stacks Red Hat Developer Experience, Brisbane Software Development Workflow Designer & Process Architect ___ SCLorg mailing list SCLorg@redhat.com https://www.redhat.com/mailman/listinfo/sclorg ___ SCLorg mailing list SCLorg@redhat.com https://www.redhat.com/mailman/listinfo/sclorg
[scl.org] Problems with Python scripts that use SCL runtimes
Hi folks, The question of the explicit "scl enable" step for using SCLs is one that's come up a few times in different contexts, and I finally sat down to document a relatively straightforward scenario that illustrates the problem: https://github.com/ncoghlan/scl-pipsi-demo#demonstrating-the-problem What the demo covers doing is: 1. Enable the Python 3.5 SCL 2. Use pipsi within that SCL to install the pygmentize script for a particular user within a virtual environment configured to use that particular runtime 3. Disable the SCL 4. Show that the installed script doesn't work due to the runtime being unable to find its shared libraries Ideally, the fact I used an SCL to install (and subsequently run) pygmentize would be an invisible implementation detail. However, at the moment, it's visible, since Python's virtual environment machinery only fixes paths for Python level imports - it doesn't touch the paths for operating system level dynamic library loading. As far as I am aware, changing the SCL binaries to set RPATH and/or RUNPATH should be sufficient to resolve the problem, at least with pipsi managed script installation (plain "--user" installations are likely to still have problems, but I'm more comfortable with a restriction saying not to use --user installs in conjunction with SCLs) Regards, Nick. -- Nick Coghlan Fedora Environments & Stacks Red Hat Developer Experience, Brisbane Software Development Workflow Designer & Process Architect ___ SCLorg mailing list SCLorg@redhat.com https://www.redhat.com/mailman/listinfo/sclorg
