Re: [Gluster-devel] Python3 build process
On 9/28/18 6:12 AM, Niels de Vos wrote: > Is it really needed to have this as an option? Instead of an option in > configure.ac, can it not be a post-install task in a Makefile.am? I don't fully understand how .pyc and .pyo files are used, or how the new-in-python3 __cache__ files are used, but they seem to be created during the build and/or the install. What does it mean to build+install the .pyc, .pyo, and __cache__ files and then go in after and whack the shebangs of the .py files? -- Kaleb ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 9/28/18 9:38 AM, Niels de Vos wrote: >>> >>> Tests should just run with "$PYTHON run-the-test.py" instead of >>> ./run-the-test.py with a #!/usr/bin/python shebang. The testing >>> framework can also find out what version of python is available. >> >> If we back up a bit here, if all shebangs are cleared, then we do not >> need anything. That is not the situation at the moment, and neither do I >> know if that state can be reached. > > Not all shebangs need to go away, only the ones for the test-cases. A > post-install hook can modify the shebangs from python3 to python2 > depending on what ./configure detected. None of the .py files in .../tests/... have shebangs (master and release-5 branch). They are all invoked with ...$PYTHON some-test.py ... All but one of them were always invoked that way. socket-as-fifo.py was the only one that was not, and that has been fixed. -- Kaleb ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On Fri, Sep 28, 2018 at 09:14:15AM -0400, Shyam Ranganathan wrote: > On 09/28/2018 09:11 AM, Niels de Vos wrote: > > On Fri, Sep 28, 2018 at 08:57:06AM -0400, Shyam Ranganathan wrote: > >> On 09/28/2018 06:12 AM, Niels de Vos wrote: > >>> On Thu, Sep 27, 2018 at 08:40:54AM -0400, Shyam Ranganathan wrote: > On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: > >> The thought is, > >> - Add a configure option "--enable-py-version-correction" to configure, > >> that is disabled by default > > "correction" implies there's something that's incorrect. How about > > "conversion" or perhaps just --enable-python2 > > > I would not like to go with --enable-python2 as that implies it is an > conscious choice with the understanding that py2 is on the box. Given > the current ability to detect and hence correct the python shebangs, I > would think we should retain it as a more detect and modify the shebangs > option name. (I am looking at this more as an option that does the right > thing implicitly than someone/tool using this checking explicitly, which > can mean different things to different people, if that makes sense) > > Now "correction" seems like an overkill, maybe "conversion"? > >>> Is it really needed to have this as an option? Instead of an option in > >>> configure.ac, can it not be a post-install task in a Makefile.am? The > >>> number of executable python scripts that get installed are minimal, so I > >>> do not expect that a lot of changes are needed for this. > >> > >> Here is how I summarize this proposal, > >> - Perform the shebang "correction" for py2 in the post install > >> - Keeps the git clone clean > >> - shebang correction occurs based on a configure time option > >> - It is not implicit but an explicit choice to correct the shebangs to > >> py2, hence we need an option either way > >> - The configure option would be "--enable-python2" > >> - Developers that need py2, can configure it as such > >> - Regression jobs that need py2, either because of the platform they > >> test against, or for py2 compliance in the future, use the said option > >> - Package builds are agnostic to these changes (currently) as they > >> decide at build time based on the platform what needs to be done. > > > > I do not think such a ./configure option is needed. configure.ac can > > find out the version that is available, and pick python3 if it has both. > > > > Tests should just run with "$PYTHON run-the-test.py" instead of > > ./run-the-test.py with a #!/usr/bin/python shebang. The testing > > framework can also find out what version of python is available. > > If we back up a bit here, if all shebangs are cleared, then we do not > need anything. That is not the situation at the moment, and neither do I > know if that state can be reached. Not all shebangs need to go away, only the ones for the test-cases. A post-install hook can modify the shebangs from python3 to python2 depending on what ./configure detected. > We also need to ensure we work against py2 and py3 for the near future, > which entails being specific in some regression job at least on the > python choice, does that correct the shebangs really depends on the > above conclusion. Ok, so if both python2 and python3 are available, and you want to run with python2, just run "python2 my-py2-test.py"? Niels ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 09/28/2018 09:11 AM, Niels de Vos wrote: > On Fri, Sep 28, 2018 at 08:57:06AM -0400, Shyam Ranganathan wrote: >> On 09/28/2018 06:12 AM, Niels de Vos wrote: >>> On Thu, Sep 27, 2018 at 08:40:54AM -0400, Shyam Ranganathan wrote: On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: >> The thought is, >> - Add a configure option "--enable-py-version-correction" to configure, >> that is disabled by default > "correction" implies there's something that's incorrect. How about > "conversion" or perhaps just --enable-python2 > I would not like to go with --enable-python2 as that implies it is an conscious choice with the understanding that py2 is on the box. Given the current ability to detect and hence correct the python shebangs, I would think we should retain it as a more detect and modify the shebangs option name. (I am looking at this more as an option that does the right thing implicitly than someone/tool using this checking explicitly, which can mean different things to different people, if that makes sense) Now "correction" seems like an overkill, maybe "conversion"? >>> Is it really needed to have this as an option? Instead of an option in >>> configure.ac, can it not be a post-install task in a Makefile.am? The >>> number of executable python scripts that get installed are minimal, so I >>> do not expect that a lot of changes are needed for this. >> >> Here is how I summarize this proposal, >> - Perform the shebang "correction" for py2 in the post install >> - Keeps the git clone clean >> - shebang correction occurs based on a configure time option >> - It is not implicit but an explicit choice to correct the shebangs to >> py2, hence we need an option either way >> - The configure option would be "--enable-python2" >> - Developers that need py2, can configure it as such >> - Regression jobs that need py2, either because of the platform they >> test against, or for py2 compliance in the future, use the said option >> - Package builds are agnostic to these changes (currently) as they >> decide at build time based on the platform what needs to be done. > > I do not think such a ./configure option is needed. configure.ac can > find out the version that is available, and pick python3 if it has both. > > Tests should just run with "$PYTHON run-the-test.py" instead of > ./run-the-test.py with a #!/usr/bin/python shebang. The testing > framework can also find out what version of python is available. If we back up a bit here, if all shebangs are cleared, then we do not need anything. That is not the situation at the moment, and neither do I know if that state can be reached. We also need to ensure we work against py2 and py3 for the near future, which entails being specific in some regression job at least on the python choice, does that correct the shebangs really depends on the above conclusion. > > >>> There do seem quite some Python files that have a shebang, but do not >>> need it (__init__.py, not executable, no __main__-like functions). This >>> should probably get reviewed as well. When those scripts get their >>> shebang removed, even fewer files need to be 'fixed-up'. >> >> I propose maintainers/component-owner take this cleanup. > > That would be ideal! > > >>> Is there a BZ or GitHub Issue that I can use to send some fixes? >> >> See: https://github.com/gluster/glusterfs/issues/411 > > Thanks, > Niels > ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On Fri, Sep 28, 2018 at 08:57:06AM -0400, Shyam Ranganathan wrote: > On 09/28/2018 06:12 AM, Niels de Vos wrote: > > On Thu, Sep 27, 2018 at 08:40:54AM -0400, Shyam Ranganathan wrote: > >> On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: > The thought is, > - Add a configure option "--enable-py-version-correction" to configure, > that is disabled by default > >>> "correction" implies there's something that's incorrect. How about > >>> "conversion" or perhaps just --enable-python2 > >>> > >> I would not like to go with --enable-python2 as that implies it is an > >> conscious choice with the understanding that py2 is on the box. Given > >> the current ability to detect and hence correct the python shebangs, I > >> would think we should retain it as a more detect and modify the shebangs > >> option name. (I am looking at this more as an option that does the right > >> thing implicitly than someone/tool using this checking explicitly, which > >> can mean different things to different people, if that makes sense) > >> > >> Now "correction" seems like an overkill, maybe "conversion"? > > Is it really needed to have this as an option? Instead of an option in > > configure.ac, can it not be a post-install task in a Makefile.am? The > > number of executable python scripts that get installed are minimal, so I > > do not expect that a lot of changes are needed for this. > > Here is how I summarize this proposal, > - Perform the shebang "correction" for py2 in the post install > - Keeps the git clone clean > - shebang correction occurs based on a configure time option > - It is not implicit but an explicit choice to correct the shebangs to > py2, hence we need an option either way > - The configure option would be "--enable-python2" > - Developers that need py2, can configure it as such > - Regression jobs that need py2, either because of the platform they > test against, or for py2 compliance in the future, use the said option > - Package builds are agnostic to these changes (currently) as they > decide at build time based on the platform what needs to be done. I do not think such a ./configure option is needed. configure.ac can find out the version that is available, and pick python3 if it has both. Tests should just run with "$PYTHON run-the-test.py" instead of ./run-the-test.py with a #!/usr/bin/python shebang. The testing framework can also find out what version of python is available. > > There do seem quite some Python files that have a shebang, but do not > > need it (__init__.py, not executable, no __main__-like functions). This > > should probably get reviewed as well. When those scripts get their > > shebang removed, even fewer files need to be 'fixed-up'. > > I propose maintainers/component-owner take this cleanup. That would be ideal! > > Is there a BZ or GitHub Issue that I can use to send some fixes? > > See: https://github.com/gluster/glusterfs/issues/411 Thanks, Niels ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 09/28/2018 12:36 AM, Kotresh Hiremath Ravishankar wrote:
> > - All regression hosts are currently py2 and so if we do not run
> the py
> > shebang correction during configure (as we do not build and test from
> > RPMS), we would be running with incorrect py3 shebangs (although this
> > seems to work, see [2]. @kotresh can we understand why?)
>
> Is it because we don't test any of the python in the regression tests?
>
> Geo-replication do have regression tests but not sure about glusterfind,
> events.
>
> Or because when we do, we invoke python scripts with `python foo.py` or
> `$PYTHON foo.py` everywhere? The shebangs are ignored when scripts are
> invoked this way.
>
> The reason why geo-rep is passing is for the same reason mentioned. Geo-rep
> python file is invoked from a c program always prefixing it with python
> as follows.
>
> python = getenv("PYTHON");
> if (!python)
> python = PYTHON;
> nargv[j++] = python;
> nargv[j++] = GSYNCD_PREFIX "/python/syncdaemon/" GSYNCD_PY;
Thank you, makes sense now.
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 09/28/2018 06:12 AM, Niels de Vos wrote: > On Thu, Sep 27, 2018 at 08:40:54AM -0400, Shyam Ranganathan wrote: >> On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: The thought is, - Add a configure option "--enable-py-version-correction" to configure, that is disabled by default >>> "correction" implies there's something that's incorrect. How about >>> "conversion" or perhaps just --enable-python2 >>> >> I would not like to go with --enable-python2 as that implies it is an >> conscious choice with the understanding that py2 is on the box. Given >> the current ability to detect and hence correct the python shebangs, I >> would think we should retain it as a more detect and modify the shebangs >> option name. (I am looking at this more as an option that does the right >> thing implicitly than someone/tool using this checking explicitly, which >> can mean different things to different people, if that makes sense) >> >> Now "correction" seems like an overkill, maybe "conversion"? > Is it really needed to have this as an option? Instead of an option in > configure.ac, can it not be a post-install task in a Makefile.am? The > number of executable python scripts that get installed are minimal, so I > do not expect that a lot of changes are needed for this. Here is how I summarize this proposal, - Perform the shebang "correction" for py2 in the post install - Keeps the git clone clean - shebang correction occurs based on a configure time option - It is not implicit but an explicit choice to correct the shebangs to py2, hence we need an option either way - The configure option would be "--enable-python2" - Developers that need py2, can configure it as such - Regression jobs that need py2, either because of the platform they test against, or for py2 compliance in the future, use the said option - Package builds are agnostic to these changes (currently) as they decide at build time based on the platform what needs to be done. > > There do seem quite some Python files that have a shebang, but do not > need it (__init__.py, not executable, no __main__-like functions). This > should probably get reviewed as well. When those scripts get their > shebang removed, even fewer files need to be 'fixed-up'. I propose maintainers/component-owner take this cleanup. > > Is there a BZ or GitHub Issue that I can use to send some fixes? See: https://github.com/gluster/glusterfs/issues/411 Shyam ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On Thu, Sep 27, 2018 at 08:40:54AM -0400, Shyam Ranganathan wrote: > On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: > >> The thought is, > >> - Add a configure option "--enable-py-version-correction" to configure, > >> that is disabled by default > > "correction" implies there's something that's incorrect. How about > > "conversion" or perhaps just --enable-python2 > > > > I would not like to go with --enable-python2 as that implies it is an > conscious choice with the understanding that py2 is on the box. Given > the current ability to detect and hence correct the python shebangs, I > would think we should retain it as a more detect and modify the shebangs > option name. (I am looking at this more as an option that does the right > thing implicitly than someone/tool using this checking explicitly, which > can mean different things to different people, if that makes sense) > > Now "correction" seems like an overkill, maybe "conversion"? Is it really needed to have this as an option? Instead of an option in configure.ac, can it not be a post-install task in a Makefile.am? The number of executable python scripts that get installed are minimal, so I do not expect that a lot of changes are needed for this. There do seem quite some Python files that have a shebang, but do not need it (__init__.py, not executable, no __main__-like functions). This should probably get reviewed as well. When those scripts get their shebang removed, even fewer files need to be 'fixed-up'. Is there a BZ or GitHub Issue that I can use to send some fixes? Thanks, Niels $ for F in $(git ls-files) ; do if ( head -n1 $F | grep -q -E '^#.+python' ) ; then echo "$F uses Python" ; fi ; done api/examples/getvolfile.py uses Python events/eventskeygen.py uses Python events/src/gf_event.py uses Python events/src/glustereventsd.py uses Python events/src/peer_eventsapi.py uses Python events/tools/eventsdash.py uses Python extras/create_new_xlator/generate_xlator.py uses Python extras/distributed-testing/distributed-test-runner.py uses Python extras/failed-tests.py uses Python extras/geo-rep/schedule_georep.py.in uses Python extras/git-branch-diff.py uses Python extras/gnfs-loganalyse.py uses Python extras/hook-scripts/S40ufo-stop.py uses Python extras/profiler/glusterfs-profiler uses Python extras/quota/quota_fsck.py uses Python extras/quota/xattr_analysis.py uses Python extras/rebalance.py uses Python extras/snap_scheduler/conf.py.in uses Python extras/snap_scheduler/gcron.py uses Python extras/snap_scheduler/snap_scheduler.py uses Python geo-replication/src/peer_georep-sshkey.py.in uses Python geo-replication/src/peer_mountbroker.in uses Python geo-replication/src/peer_mountbroker.py.in uses Python geo-replication/syncdaemon/changelogagent.py uses Python geo-replication/syncdaemon/conf.py.in uses Python geo-replication/syncdaemon/gsyncd.py uses Python geo-replication/syncdaemon/gsyncdstatus.py uses Python geo-replication/tests/__init__.py uses Python geo-replication/tests/unit/__init__.py uses Python geo-replication/tests/unit/test_gsyncdstatus.py uses Python geo-replication/tests/unit/test_syncdutils.py uses Python libglusterfs/src/gen-defaults.py uses Python libglusterfs/src/generator.py uses Python tools/gfind_missing_files/gfid_to_path.py uses Python tools/glusterfind/S57glusterfind-delete-post.py uses Python tools/glusterfind/glusterfind.in uses Python tools/glusterfind/src/brickfind.py uses Python tools/glusterfind/src/changelog.py uses Python tools/glusterfind/src/main.py uses Python tools/glusterfind/src/nodeagent.py uses Python xlators/experimental/fdl/src/gen_dumper.py uses Python xlators/experimental/fdl/src/gen_fdl.py uses Python xlators/experimental/fdl/src/gen_recon.py uses Python xlators/experimental/jbr-client/src/gen-fops.py uses Python xlators/experimental/jbr-server/src/gen-fops.py uses Python xlators/features/changelog/lib/examples/python/changes.py uses Python xlators/features/cloudsync/src/cloudsync-fops-c.py uses Python xlators/features/cloudsync/src/cloudsync-fops-h.py uses Python xlators/features/glupy/src/__init__.py.in uses Python xlators/features/utime/src/utime-gen-fops-c.py uses Python xlators/features/utime/src/utime-gen-fops-h.py uses Python ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On Thu, Sep 27, 2018 at 5:38 PM Kaleb S. KEITHLEY
wrote:
> On 9/26/18 8:28 PM, Shyam Ranganathan wrote:
> > Hi,
> >
> > With the introduction of default python 3 shebangs and the change in
> > configure.ac to correct these to py2 if the build is being attempted on
> > a machine that does not have py3, there are a couple of issues
> > uncovered. Here is the plan to fix the same, suggestions welcome.
> >
> > Issues:
> > - A configure job is run when creating the dist tarball, and this runs
> > on non py3 platforms, hence changing the dist tarball to basically have
> > py2 shebangs, as a result the release-new build job always outputs py
> > files with the py2 shebang. See tarball in [1]
> >
> > - All regression hosts are currently py2 and so if we do not run the py
> > shebang correction during configure (as we do not build and test from
> > RPMS), we would be running with incorrect py3 shebangs (although this
> > seems to work, see [2]. @kotresh can we understand why?)
>
> Is it because we don't test any of the python in the regression tests?
>
> Geo-replication do have regression tests but not sure about glusterfind,
events.
Or because when we do, we invoke python scripts with `python foo.py` or
> `$PYTHON foo.py` everywhere? The shebangs are ignored when scripts are
> invoked this way.
>
The reason why geo-rep is passing is for the same reason mentioned. Geo-rep
python file is invoked from a c program always prefixing it with python as
follows.
python = getenv("PYTHON");
if (!python)
python = PYTHON;
nargv[j++] = python;
nargv[j++] = GSYNCD_PREFIX "/python/syncdaemon/" GSYNCD_PY;
>
> > Plan to address the above is detailed in this bug [3].
> >
> > The thought is,
> > - Add a configure option "--enable-py-version-correction" to configure,
> > that is disabled by default
>
> "correction" implies there's something that's incorrect. How about
> "conversion" or perhaps just --enable-python2
>
> >
> > - All regression jobs will run with the above option, and hence this
> > will correct the py shebangs in the regression machines. In the future
> > as we run on both py2 and py3 machines, this will run with the right
> > python shebangs on these machines.
> >
> > - The packaging jobs will now run the py version detection and shebang
> > correction during actual build and packaging, Kaleb already has put up a
> > patch for the same [2].
> >
> > Thoughts?
> >
>
> Also note that until --enable-whatever is added to configure(.ac), if
> you're building and testing any of the python bits on RHEL or CentOS
> you'll need to convert the shebangs. Perhaps the easiest way to do that
> now (master branch and release-5 branch) is to build+install rpms.
>
> If you're currently doing
>
> `git clone; ./autogen.sh; ./configure; make; make install`
>
> then change that to
>
> `git clone; ./autogen.sh; ./configure; make -C extras/LinuxRPMS
> glusterrpms`
>
> and then yum install those rpms. The added advantage is that it's easier
> to remove rpms than anything installed with `make install`.
>
> If you're developing on Fedora (hopefully 27 or later) or Debian or
> Ubuntu you don't need to do anything different as they all have python3.
>
> --
>
> Kaleb
> ___
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>
--
Thanks and Regards,
Kotresh H R
___
Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 9/27/18 8:57 AM, Kaleb S. KEITHLEY wrote: > On 9/27/18 8:40 AM, Shyam Ranganathan wrote: >> On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: The thought is, - Add a configure option "--enable-py-version-correction" to configure, that is disabled by default >>> "correction" implies there's something that's incorrect. How about >>> "conversion" or perhaps just --enable-python2 >>> >> >> I would not like to go with --enable-python2 as that implies it is an >> conscious choice with the understanding that py2 is on the box. Given >> the current ability to detect and hence correct the python shebangs, I >> would think we should retain it as a more detect and modify the shebangs >> option name. (I am looking at this more as an option that does the right >> thing implicitly than someone/tool using this checking explicitly, which >> can mean different things to different people, if that makes sense) >> >> Now "correction" seems like an overkill, maybe "conversion"? >> > > I guess I don't really care what the option is called. > > The only conversion is _to_ python2 and the only place it ever _needs_ > to be done is RHEL < 8 and eventually CentOS < 8. that s/b ... and CentOS (eventually CentOS < 8). > > (You could argue that python3 -> python3 is a conversion.) Are you > saying that if you do `./configure --enable-py-version-conversion` on, > e.g. a Fedora < 30 box, that the shebangs will still be > #!/usr/bin/python3 because python3 is on the box? I think that would be > surprising. > > I can imagine that someone might want to convert them on Fedora < 30 and > Debian/Ubuntu/etc because they want to _test_ that the python bits still > work with python2. (Or they actually want to use python2 over python3 > for some reason.) > > -- > > Kaleb > ___ > Gluster-devel mailing list > Gluster-devel@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-devel > ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 9/27/18 8:40 AM, Shyam Ranganathan wrote: > On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: >>> The thought is, >>> - Add a configure option "--enable-py-version-correction" to configure, >>> that is disabled by default >> "correction" implies there's something that's incorrect. How about >> "conversion" or perhaps just --enable-python2 >> > > I would not like to go with --enable-python2 as that implies it is an > conscious choice with the understanding that py2 is on the box. Given > the current ability to detect and hence correct the python shebangs, I > would think we should retain it as a more detect and modify the shebangs > option name. (I am looking at this more as an option that does the right > thing implicitly than someone/tool using this checking explicitly, which > can mean different things to different people, if that makes sense) > > Now "correction" seems like an overkill, maybe "conversion"? > I guess I don't really care what the option is called. The only conversion is _to_ python2 and the only place it ever _needs_ to be done is RHEL < 8 and eventually CentOS < 8. (You could argue that python3 -> python3 is a conversion.) Are you saying that if you do `./configure --enable-py-version-conversion` on, e.g. a Fedora < 30 box, that the shebangs will still be #!/usr/bin/python3 because python3 is on the box? I think that would be surprising. I can imagine that someone might want to convert them on Fedora < 30 and Debian/Ubuntu/etc because they want to _test_ that the python bits still work with python2. (Or they actually want to use python2 over python3 for some reason.) -- Kaleb ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 09/27/2018 08:07 AM, Kaleb S. KEITHLEY wrote: >> The thought is, >> - Add a configure option "--enable-py-version-correction" to configure, >> that is disabled by default > "correction" implies there's something that's incorrect. How about > "conversion" or perhaps just --enable-python2 > I would not like to go with --enable-python2 as that implies it is an conscious choice with the understanding that py2 is on the box. Given the current ability to detect and hence correct the python shebangs, I would think we should retain it as a more detect and modify the shebangs option name. (I am looking at this more as an option that does the right thing implicitly than someone/tool using this checking explicitly, which can mean different things to different people, if that makes sense) Now "correction" seems like an overkill, maybe "conversion"? ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] Python3 build process
On 9/26/18 8:28 PM, Shyam Ranganathan wrote: > Hi, > > With the introduction of default python 3 shebangs and the change in > configure.ac to correct these to py2 if the build is being attempted on > a machine that does not have py3, there are a couple of issues > uncovered. Here is the plan to fix the same, suggestions welcome. > > Issues: > - A configure job is run when creating the dist tarball, and this runs > on non py3 platforms, hence changing the dist tarball to basically have > py2 shebangs, as a result the release-new build job always outputs py > files with the py2 shebang. See tarball in [1] > > - All regression hosts are currently py2 and so if we do not run the py > shebang correction during configure (as we do not build and test from > RPMS), we would be running with incorrect py3 shebangs (although this > seems to work, see [2]. @kotresh can we understand why?) Is it because we don't test any of the python in the regression tests? Or because when we do, we invoke python scripts with `python foo.py` or `$PYTHON foo.py` everywhere? The shebangs are ignored when scripts are invoked this way. > > Plan to address the above is detailed in this bug [3]. > > The thought is, > - Add a configure option "--enable-py-version-correction" to configure, > that is disabled by default "correction" implies there's something that's incorrect. How about "conversion" or perhaps just --enable-python2 > > - All regression jobs will run with the above option, and hence this > will correct the py shebangs in the regression machines. In the future > as we run on both py2 and py3 machines, this will run with the right > python shebangs on these machines. > > - The packaging jobs will now run the py version detection and shebang > correction during actual build and packaging, Kaleb already has put up a > patch for the same [2]. > > Thoughts? > Also note that until --enable-whatever is added to configure(.ac), if you're building and testing any of the python bits on RHEL or CentOS you'll need to convert the shebangs. Perhaps the easiest way to do that now (master branch and release-5 branch) is to build+install rpms. If you're currently doing `git clone; ./autogen.sh; ./configure; make; make install` then change that to `git clone; ./autogen.sh; ./configure; make -C extras/LinuxRPMS glusterrpms` and then yum install those rpms. The added advantage is that it's easier to remove rpms than anything installed with `make install`. If you're developing on Fedora (hopefully 27 or later) or Debian or Ubuntu you don't need to do anything different as they all have python3. -- Kaleb ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
[Gluster-devel] Python3 build process
Hi, With the introduction of default python 3 shebangs and the change in configure.ac to correct these to py2 if the build is being attempted on a machine that does not have py3, there are a couple of issues uncovered. Here is the plan to fix the same, suggestions welcome. Issues: - A configure job is run when creating the dist tarball, and this runs on non py3 platforms, hence changing the dist tarball to basically have py2 shebangs, as a result the release-new build job always outputs py files with the py2 shebang. See tarball in [1] - All regression hosts are currently py2 and so if we do not run the py shebang correction during configure (as we do not build and test from RPMS), we would be running with incorrect py3 shebangs (although this seems to work, see [2]. @kotresh can we understand why?) Plan to address the above is detailed in this bug [3]. The thought is, - Add a configure option "--enable-py-version-correction" to configure, that is disabled by default - All regression jobs will run with the above option, and hence this will correct the py shebangs in the regression machines. In the future as we run on both py2 and py3 machines, this will run with the right python shebangs on these machines. - The packaging jobs will now run the py version detection and shebang correction during actual build and packaging, Kaleb already has put up a patch for the same [2]. Thoughts? Shyam [1] Release tarball: https://build.gluster.org/job/release-new/69/ [2] Patch that defaults to py3 in regression and passes regressions: https://review.gluster.org/c/glusterfs/+/21266 [3] Infra bug to change regression jobs: https://bugzilla.redhat.com/show_bug.cgi?id=1633425 ___ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
