Re: [PATCH 2 of 2] setup: prevent setuptools from laying an egg

[Matt Harbison]

On Tue, 09 May 2017 14:23:18 -0400, Augie Fackler  wrote:

On Mon, May 8, 2017 at 12:39 AM, Matt Harbison   
wrote:

# HG changeset patch
# User Matt Harbison 
# Date 1494214143 14400
#  Sun May 07 23:29:03 2017 -0400
# Node ID 707683d56e5cbd3fe3453ddab9a57222b998f2af
# Parent  36d9a659b9d76837faaf73fde3f5c5455231c2f9
setup: prevent setuptools from laying an egg


I just ran this patch by dstufft (the pip maintainer), and he strongly
suspects (without having tested) that this will break `pip list` for
Mercurial. Also, he said this:

14:21 < dstufft> durin42: I think if you do ``mkdir -p src && mv
mercurial src/``
 and patch setup.py to add ``package_dir={"": "src"},``  
it'll

 resolve that for you
14:22 < dstufft> durin42: the problem you're going to run into, is stuff  
depends
 on that .egg-info, even though it says egg in the name  
still,
 that's still a part of modern packaging, and the reason  
why it
 was picking up local-source-tree hg is likely going to  
be

 because python adds ``.`` to sys.path at the very front

So we should probably find some other workaround here. :(


Bafflingly, an empty install class lets the test work, but does not  
prevent the generation of .egg-info:


diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -463,10 +463,11 @@
 return os.path.join(self.build_temp, dir, 'hg.exe')

 class hginstall(install):
-def get_sub_commands(self):
-# Screen out egg related commands to prevent egg generation
-excl = set(['install_egg_info', 'bdist_egg'])
-return filter(lambda x: x not in excl,  
install.get_sub_commands(self))

+pass
+##def get_sub_commands(self):
+### Screen out egg related commands to prevent egg generation
+##excl = set(['install_egg_info', 'bdist_egg'])
+##return filter(lambda x: x not in excl,  
install.get_sub_commands(self))


 class hginstalllib(install_lib):
 '''
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2464,7 +2464,7 @@
b' build %(compiler)s --build-base="%(base)s"'
b' install --force --prefix="%(prefix)s"'
b' --install-lib="%(libdir)s"'
-   b' --install-scripts="%(bindir)s" %(nohome)s >%(logfile)s  
2>&1'
+   b' --install-scripts="%(bindir)s" %(nohome)s' #  

%(logfile)s 2>&1'

% {b'exe': exe, b'pure': pure,
   b'compiler': compiler,
   b'base': os.path.join(self._hgtmp, b"build"),

Alternately, adding --local to that test will cause the error to be  
skipped [1].  (It ends up using hg from the source tree.)  But I didn't do  
that, thinking that undermining the check was the wrong way to go.  I  
assume if we keep generating eggs, `make clean` should get rid of it.


Regarding your previous email about finding a way to not 'cd' back to the  
source tree, I don't think that's what is happening.  PYTHONPATH when hg  
is installed for the internal test is:


C:\Users\Matt\AppData\Local\Temp\hgtests.ijjckm\install\lib\python;$TESTTMP;c:\Users\Matt\Projects\hg\tests;c:\Users\Matt\Projects\hg;c:\Users\Matt\Projects\hg\tests;c:\Users\Matt\Projects\hg\tests

(Interesting, that repetition at the end.)

self._installdir for the internal test matches that first path.  I assume  
that when the egg is installed, there's nothing found there, and it simply  
falls back to the source tree.  When I use the empty install class and  
print 'actualhg' in that check, it _does_ use the temp installed package  
instead of falling back to the source tree.  So while the empty class is a  
bit of black magic, I think it keeps us where we are without setuptools.



[1]  
https://www.mercurial-scm.org/repo/hg/file/247bb7a2c492/tests/run-tests.py#l2553





Previously, test-hghave.t was failing on Windows (and on Linux if
$FORCE_SETUPTOOLS was set) with the following:

  --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
  +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
  @@ -19,7 +19,11 @@
 >   foo
 > EOF
 $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
  +  warning: Testing with unexpected mercurial lib:  
c:\Users\Matt\Projects\hg\mercurial

  +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
 .
  +  warning: Tested with unexpected mercurial lib:  
c:\Users\Matt\Projects\hg\mercurial

  +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)

Also, `make install` was creating an *.egg-info in the local Mercurial  
source

tree, which `make clean` wasn't deleting.

Someone familiar with setuptools should take a close look at this.  I  
originally
registered a subclass of 'install_egg_info' with an empty run(), which  
took care
of the egg-info generated by `make install`.  Looking at the 

Re: [PATCH 2 of 2] setup: prevent setuptools from laying an egg

[Augie Fackler]

On Mon, May 8, 2017 at 12:39 AM, Matt Harbison  wrote:
> Previously, test-hghave.t was failing on Windows (and on Linux if
> $FORCE_SETUPTOOLS was set) with the following:
>
>   --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
>   +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
>   @@ -19,7 +19,11 @@
>  >   foo
>  > EOF
>  $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
>   +  warning: Testing with unexpected mercurial lib: 
> c:\Users\Matt\Projects\hg\mercurial
>   +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
>  .
>   +  warning: Tested with unexpected mercurial lib: 
> c:\Users\Matt\Projects\hg\mercurial
>   +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)


Looking at this particular failure some more, we should probably just
come up with a way to run this test without having to cd back into the
original source tree.
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] setup: prevent setuptools from laying an egg

[Augie Fackler]

On Mon, May 8, 2017 at 12:39 AM, Matt Harbison  wrote:
> # HG changeset patch
> # User Matt Harbison 
> # Date 1494214143 14400
> #  Sun May 07 23:29:03 2017 -0400
> # Node ID 707683d56e5cbd3fe3453ddab9a57222b998f2af
> # Parent  36d9a659b9d76837faaf73fde3f5c5455231c2f9
> setup: prevent setuptools from laying an egg

I just ran this patch by dstufft (the pip maintainer), and he strongly
suspects (without having tested) that this will break `pip list` for
Mercurial. Also, he said this:

14:21 < dstufft> durin42: I think if you do ``mkdir -p src && mv
mercurial src/``
 and patch setup.py to add ``package_dir={"": "src"},`` it'll
 resolve that for you
14:22 < dstufft> durin42: the problem you're going to run into, is stuff depends
 on that .egg-info, even though it says egg in the name still,
 that's still a part of modern packaging, and the reason why it
 was picking up local-source-tree hg is likely going to be
 because python adds ``.`` to sys.path at the very front

So we should probably find some other workaround here. :(

>
> Previously, test-hghave.t was failing on Windows (and on Linux if
> $FORCE_SETUPTOOLS was set) with the following:
>
>   --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
>   +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
>   @@ -19,7 +19,11 @@
>  >   foo
>  > EOF
>  $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
>   +  warning: Testing with unexpected mercurial lib: 
> c:\Users\Matt\Projects\hg\mercurial
>   +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
>  .
>   +  warning: Tested with unexpected mercurial lib: 
> c:\Users\Matt\Projects\hg\mercurial
>   +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
>
> Also, `make install` was creating an *.egg-info in the local Mercurial source
> tree, which `make clean` wasn't deleting.
>
> Someone familiar with setuptools should take a close look at this.  I 
> originally
> registered a subclass of 'install_egg_info' with an empty run(), which took 
> care
> of the egg-info generated by `make install`.  Looking at the output of 
> setup.py
> as run by run-tests._installhg(), I saw that 'bdist_egg' was being run.  I
> registered a similar empty class for that command, but the egg was still
> produced.  But when I registered this 'install' subclass with nothing but a
> run() that called the super class run, 'bdist_egg' was no longer being called,
> and the test worked.  Subsequently dropping everything except the 'install'
> subclass with a minimal delegating run() allowed the test to run (though the
> setup log had output from running 'install_egg_info').
>
> To be on the safe side (and to avoid output about eggs when the empty 
> subclasses
> would have prevented any egg processing), I added a filter to 
> get_sub_commands()
> to filter out the egg related stuff.  Unlike the others, 'bdist_egg' is part 
> of
> setuptools, so registering a subclass and still working without setuptools
> probably isn't very clean.
>
> diff --git a/setup.py b/setup.py
> --- a/setup.py
> +++ b/setup.py
> @@ -77,6 +77,7 @@
>  from distutils.command.build_ext import build_ext
>  from distutils.command.build_py import build_py
>  from distutils.command.build_scripts import build_scripts
> +from distutils.command.install import install
>  from distutils.command.install_lib import install_lib
>  from distutils.command.install_scripts import install_scripts
>  from distutils.spawn import spawn, find_executable
> @@ -461,6 +462,12 @@
>  dir = os.path.dirname(self.get_ext_fullpath('dummy'))
>  return os.path.join(self.build_temp, dir, 'hg.exe')
>
> +class hginstall(install):
> +def get_sub_commands(self):
> +# Screen out egg related commands to prevent egg generation
> +excl = set(['install_egg_info', 'bdist_egg'])
> +return filter(lambda x: x not in excl, 
> install.get_sub_commands(self))
> +
>  class hginstalllib(install_lib):
>  '''
>  This is a specialization of install_lib that replaces the copy_file used
> @@ -572,6 +579,7 @@
>  'build_py': hgbuildpy,
>  'build_scripts': hgbuildscripts,
>  'build_hgextindex': buildhgextindex,
> +'install': hginstall,
>  'install_lib': hginstalllib,
>  'install_scripts': hginstallscripts,
>  'build_hgexe': buildhgexe,
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

Re: [PATCH 2 of 2] setup: prevent setuptools from laying an egg

[Gregory Szorc]

On Sun, May 7, 2017 at 9:39 PM, Matt Harbison  wrote:

> # HG changeset patch
> # User Matt Harbison 
> # Date 1494214143 14400
> #  Sun May 07 23:29:03 2017 -0400
> # Node ID 707683d56e5cbd3fe3453ddab9a57222b998f2af
> # Parent  36d9a659b9d76837faaf73fde3f5c5455231c2f9
> setup: prevent setuptools from laying an egg
>

This patch seems reasonable. I'm, uh, actually quite surprised setuptools
doesn't have an easier way to disable egg generation considering eggs have
fallen out of favor for Python packaging. Who knows.


>
> Previously, test-hghave.t was failing on Windows (and on Linux if
> $FORCE_SETUPTOOLS was set) with the following:
>
>   --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
>   +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
>   @@ -19,7 +19,11 @@
>  >   foo
>  > EOF
>  $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
>   +  warning: Testing with unexpected mercurial lib:
> c:\Users\Matt\Projects\hg\mercurial
>   +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
>  .
>   +  warning: Tested with unexpected mercurial lib:
> c:\Users\Matt\Projects\hg\mercurial
>   +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
>
> Also, `make install` was creating an *.egg-info in the local Mercurial
> source
> tree, which `make clean` wasn't deleting.
>
> Someone familiar with setuptools should take a close look at this.  I
> originally
> registered a subclass of 'install_egg_info' with an empty run(), which
> took care
> of the egg-info generated by `make install`.  Looking at the output of
> setup.py
> as run by run-tests._installhg(), I saw that 'bdist_egg' was being run.  I
> registered a similar empty class for that command, but the egg was still
> produced.  But when I registered this 'install' subclass with nothing but a
> run() that called the super class run, 'bdist_egg' was no longer being
> called,
> and the test worked.  Subsequently dropping everything except the 'install'
> subclass with a minimal delegating run() allowed the test to run (though
> the
> setup log had output from running 'install_egg_info').
>
> To be on the safe side (and to avoid output about eggs when the empty
> subclasses
> would have prevented any egg processing), I added a filter to
> get_sub_commands()
> to filter out the egg related stuff.  Unlike the others, 'bdist_egg' is
> part of
> setuptools, so registering a subclass and still working without setuptools
> probably isn't very clean.
>
> diff --git a/setup.py b/setup.py
> --- a/setup.py
> +++ b/setup.py
> @@ -77,6 +77,7 @@
>  from distutils.command.build_ext import build_ext
>  from distutils.command.build_py import build_py
>  from distutils.command.build_scripts import build_scripts
> +from distutils.command.install import install
>  from distutils.command.install_lib import install_lib
>  from distutils.command.install_scripts import install_scripts
>  from distutils.spawn import spawn, find_executable
> @@ -461,6 +462,12 @@
>  dir = os.path.dirname(self.get_ext_fullpath('dummy'))
>  return os.path.join(self.build_temp, dir, 'hg.exe')
>
> +class hginstall(install):
> +def get_sub_commands(self):
> +# Screen out egg related commands to prevent egg generation
> +excl = set(['install_egg_info', 'bdist_egg'])
> +return filter(lambda x: x not in excl,
> install.get_sub_commands(self))
> +
>  class hginstalllib(install_lib):
>  '''
>  This is a specialization of install_lib that replaces the copy_file
> used
> @@ -572,6 +579,7 @@
>  'build_py': hgbuildpy,
>  'build_scripts': hgbuildscripts,
>  'build_hgextindex': buildhgextindex,
> +'install': hginstall,
>  'install_lib': hginstalllib,
>  'install_scripts': hginstallscripts,
>  'build_hgexe': buildhgexe,
> ___
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel

[PATCH 2 of 2] setup: prevent setuptools from laying an egg

[Matt Harbison]

# HG changeset patch
# User Matt Harbison 
# Date 1494214143 14400
#  Sun May 07 23:29:03 2017 -0400
# Node ID 707683d56e5cbd3fe3453ddab9a57222b998f2af
# Parent  36d9a659b9d76837faaf73fde3f5c5455231c2f9
setup: prevent setuptools from laying an egg

Previously, test-hghave.t was failing on Windows (and on Linux if
$FORCE_SETUPTOOLS was set) with the following:

  --- c:/Users/Matt/Projects/hg/tests/test-hghave.t
  +++ c:/Users/Matt/Projects/hg/tests/test-hghave.t.err
  @@ -19,7 +19,11 @@
 >   foo
 > EOF
 $ run-tests.py $HGTEST_RUN_TESTS_PURE test-hghaveaddon.t
  +  warning: Testing with unexpected mercurial lib: 
c:\Users\Matt\Projects\hg\mercurial
  +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)
 .
  +  warning: Tested with unexpected mercurial lib: 
c:\Users\Matt\Projects\hg\mercurial
  +   (expected ...\hgtests.mu9rou\install\lib\python\mercurial)

Also, `make install` was creating an *.egg-info in the local Mercurial source
tree, which `make clean` wasn't deleting.

Someone familiar with setuptools should take a close look at this.  I originally
registered a subclass of 'install_egg_info' with an empty run(), which took care
of the egg-info generated by `make install`.  Looking at the output of setup.py
as run by run-tests._installhg(), I saw that 'bdist_egg' was being run.  I
registered a similar empty class for that command, but the egg was still
produced.  But when I registered this 'install' subclass with nothing but a
run() that called the super class run, 'bdist_egg' was no longer being called,
and the test worked.  Subsequently dropping everything except the 'install'
subclass with a minimal delegating run() allowed the test to run (though the
setup log had output from running 'install_egg_info').

To be on the safe side (and to avoid output about eggs when the empty subclasses
would have prevented any egg processing), I added a filter to get_sub_commands()
to filter out the egg related stuff.  Unlike the others, 'bdist_egg' is part of
setuptools, so registering a subclass and still working without setuptools
probably isn't very clean.

diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -77,6 +77,7 @@
 from distutils.command.build_ext import build_ext
 from distutils.command.build_py import build_py
 from distutils.command.build_scripts import build_scripts
+from distutils.command.install import install
 from distutils.command.install_lib import install_lib
 from distutils.command.install_scripts import install_scripts
 from distutils.spawn import spawn, find_executable
@@ -461,6 +462,12 @@
 dir = os.path.dirname(self.get_ext_fullpath('dummy'))
 return os.path.join(self.build_temp, dir, 'hg.exe')
 
+class hginstall(install):
+def get_sub_commands(self):
+# Screen out egg related commands to prevent egg generation
+excl = set(['install_egg_info', 'bdist_egg'])
+return filter(lambda x: x not in excl, install.get_sub_commands(self))
+
 class hginstalllib(install_lib):
 '''
 This is a specialization of install_lib that replaces the copy_file used
@@ -572,6 +579,7 @@
 'build_py': hgbuildpy,
 'build_scripts': hgbuildscripts,
 'build_hgextindex': buildhgextindex,
+'install': hginstall,
 'install_lib': hginstalllib,
 'install_scripts': hginstallscripts,
 'build_hgexe': buildhgexe,
___
Mercurial-devel mailing list
Mercurial-devel@mercurial-scm.org
https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel