[Distutils] Re: Packaging Advice for EFF's Certbot

[Matt Joyce]

It's generally easiest to find a maintainer who has an established track
record to adopt the project.

On Sat, Jul 28, 2018 at 11:53 AM,  wrote:

> Nathaniel Smith wrote:
> > I love Debian. I've been a loyal user for twenty years. But Debian is
> > really bad at coping with software that needs to react quickly to
> changing
> > external conditions, or that depends on a tight feedback loop between
> users
> > and developers. (Indeed, Debian's whole value-add is to insert themselves
> > as a buffer between users and developers, which is great in some
> situations
> > but terrible in others.)
> >
> > Maybe if certbot worked hard enough they could arrange to get special
> > treatment like Firefox or clamav or something, but in their position I
> > would see this as a total non-starter. Even if they did somehow manage to
> > navigate Debian's politics, they'd still have to go and repeat the
> process
> > for Redhat, SuSE, Ubuntu, Gentoo, ...
>
> I wouldn't be too put off by the idea of Debian politics. Certbot should
> be a good fit for stable-updates:
> https://wiki.debian.org/StableUpdates
> under the "Packages that need to be current to be useful" criterion listed
> at https://www.debian.org/News/2011/20110215
>
> -M-
> --
> Distutils-SIG mailing list -- distutils-sig@python.org
> To unsubscribe send an email to distutils-sig-le...@python.org
> https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
> Message archived at https://mail.python.org/mm3/
> archives/list/distutils-sig@python.org/message/
> EGMGR2HFKJOPANXPORLSLNDRTYVB3FOM/
>
--
Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-le...@python.org
https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/
Message archived at 
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/Y4OBW44DFNHDHZ6ZI524FDAYXGDC4J4U/

Re: [Distutils] PEP 517: Bootstrapping setuptools

[Matt Joyce]

venvs within venvs... terrifying concept.

On Tue, Aug 22, 2017 at 11:02 AM, Jim Fulton  wrote:

>
>
> On Tue, Aug 22, 2017 at 9:23 AM, Daniel Holth  wrote:
>
>> Isn't this a special case of needing . on sys.path when building a
>> package? Then the same version of setuptools that is being packaged builds
>> itself.
>>
> No. The issue for us it wasn't setuptools itself, but it's dependencies
> and those dependencies conflicted with dependencies of of packages we were
> installing *and* those packages importing these dependences (indirectly) in
> their setup scripts.  Setup scripts that import the thing they're about to
> install, generally to get the version :(, is something I'd love to see go
> away.
>
> Jim
>
> --
> Jim Fulton
> http://jimfulton.info
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Fatal error "Python.h"

[Matt Joyce]

Look for a python-dev or python-devel package.

On Jun 26, 2017 6:36 AM, "Rick"  wrote:

> Hi
> Can anyone help?
> I’m pretty new to Python and Raspberry Pi.
> When running PIP, I seem to get fatal error:
>
> Python.h: No such file or directory
>
> include “Python.h”
>
> compilation terminated
>
> I get this when attempting to download both *Cython* and *scikit-image*.
> So I’m assuming it is some absent file or path reference on the Raspberry
> Pi installation I have, rather than in the distribution package?
>
> What is this file “Python.h” that it is trying to include?
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Can u provide metadata about the PyPI

[Matt Joyce]

https://kirankoduru.github.io/python/pypi-stats.html

this may be relevant to your interests.  some of the pypi info is available
on google big table.

On Sat, Jun 24, 2017 at 5:02 AM, Priya Lakshmi 
wrote:

> Respected Python expert
>
> I just want a help. Can u provide us the metadata available about the
> modules in PyPI giant database. For example, we want the number of packages
> available, the number of downloads for each package, etc
> So we can think of doing some good maintenance work.
>
> thank you.
> G Priyalakshmi
> Assistant Professor (Selection Grade)
> Department of Applied Mathematics and Computational Sciences
> PSG College of Technology
> Coimbatore
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Malicious packages on PyPI

[Matt Joyce]

I was more pushing for the transitive trust element than signing.  That
being said, any signing at all would be progress.

On Jun 1, 2017 9:07 PM, "Donald Stufft" <don...@stufft.io> wrote:


On Jun 1, 2017, at 8:15 PM, Matt Joyce <m...@nycresistor.com> wrote:

Or start doing signed pgp for package maintainers and build a transitive
trust model.



PGP is not useful for our use case except as a generic crypto primitive,
and there are better generic crypto primitives out there. See
https://caremad.io/posts/2013/07/packaging-signing-not-holy-grail/


—
Donald Stufft
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Malicious packages on PyPI

[Matt Joyce]

Force packages to match their higher level import namespace in future major
Python versions and PEP it.

On Jun 1, 2017 7:37 PM, "Noah Kantrowitz"  wrote:

>
> > On Jun 1, 2017, at 4:00 PM, Nick Timkovich 
> wrote:
> >
> > This issue was also brought up in January at
> https://github.com/pypa/pypi-legacy/issues/585 then just as after the
> initial "typosquatting PyPI" report (June 2016) it's met with resounding
> silence. Attacking the messenger doesn't seem like a winning move from a
> security standpoint.
> >
> > Can we come up with a plan to address the underlying issue and protect
> users?
>
> If you have a systemic solution I'm sure we would love to hear it :)
>
> --Noah
>
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Malicious packages on PyPI

[Matt Joyce]

Or start doing signed pgp for package maintainers and build a transitive
trust model.

On Jun 1, 2017 8:14 PM, wrote:

Force packages to match their higher level import namespace in future major
Python versions and PEP it.

On Jun 1, 2017 7:37 PM, "Noah Kantrowitz"  wrote:


> On Jun 1, 2017, at 4:00 PM, Nick Timkovich 
wrote:
>
> This issue was also brought up in January at https://github.com/pypa/pypi-
legacy/issues/585 then just as after the initial "typosquatting PyPI"
report (June 2016) it's met with resounding silence. Attacking the
messenger doesn't seem like a winning move from a security standpoint.
>
> Can we come up with a plan to address the underlying issue and protect
users?

If you have a systemic solution I'm sure we would love to hear it :)

--Noah



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Malicious packages on PyPI

[Matt Joyce]

I mean the easy attack vector is find a package where the package name does
not match the import namespace.  If the import namespace has no
corresponding package in pypi... register it.

Anyone who blind tries to grab a dependency will grab your module instead
of the one they want.

Horrible to do.  But that's the attack vector.

On Thu, Jun 1, 2017 at 3:31 PM, Xavier Fernandez <xav.fernan...@gmail.com>
wrote:

> This makes me remember https://hackernoon.com/building-a-botnet-on-pypi-
> be1ad280b8d6 on a related note.
>
> On Thu, Jun 1, 2017 at 7:40 PM, Thomas Kluyver <tho...@kluyver.me.uk>
> wrote:
>
>> On Thu, Jun 1, 2017, at 06:32 PM, Matt Joyce wrote:
>>
>> It's basically a test dummy package that reports users who have ran that
>> package template.
>>
>>
>> That's what I thought, but all the code to do the upload seems to have
>> been removed before s/he built those packages. Now it's just a harmless
>> warning, unless I'm missing something.
>>
>> https://github.com/fate0/cookiecutter-evilpy-package/commit/
>> a3ed1e1e060748b0444158ea3bc569dfbf57645e
>>
>> the site referenced lists the package name that the user ran to get
>> posted to the site.   there appear to be many packages in pypi that are
>> built off this fatezero template.
>>
>>
>> There *appear* to be, but I checked several of the names listed there,
>> and they're not on PyPI:
>>
>> https://pypi.python.org/pypi/tkinter
>> https://pypi.python.org/pypi/memcached
>> https://pypi.python.org/pypi/vtk
>> https://pypi.python.org/pypi/python-dev
>> https://pypi.python.org/pypi/opencv
>>
>> So I wonder if the data is fake. Or maybe they were already taken down?
>> Or the installations are real, but not using those names.
>>
>> pypi is not a very good package management solution.  most folks I advise
>> to build from pypi in CI/CD but push to production via a real package
>> management solution such as apt or yum.  always double check sources coming
>> from the internet.
>>
>>
>> It's an open repository that anyone can upload to. That has its drawbacks
>> and its advantages.
>>
>>
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Malicious packages on PyPI

[Matt Joyce]

https://github.com/fate0/cookiecutter-evilpy-package/tree/master/%7B%7Bcookiecutter.package_name%7D%7D

that's the package repo on github.

It's basically a test dummy package that reports users who have ran that
package template.

the site referenced lists the package name that the user ran to get posted
to the site.   there appear to be many packages in pypi that are built off
this fatezero template.

it is non destructive... as a test payload.  but the method used is
obviously highly successful as an attack vector.  there may be more
nefarious packages already in pypi.

pypi is not a very good package management solution.  most folks I advise
to build from pypi in CI/CD but push to production via a real package
management solution such as apt or yum.  always double check sources coming
from the internet.

-Matt

On Thu, Jun 1, 2017 at 1:24 PM, Thomas Kluyver  wrote:

> On closer examination, those packages do not actually appear to upload
> any information - they seem to be empty packages placed there to serve
> as a warning.
>
> It's not clear to me whether the data on the fatezero.org website is
> from other packages which really do upload data, or if it's fake.
>
> On Thu, Jun 1, 2017, at 06:18 PM, Thomas Kluyver wrote:
> > Are we aware of this?
> > http://evilpackage.fatezero.org/
> >
> > I recall there were a couple of these before which were taken down, but
> > someone appears to have made a cookiecutter template so you can very
> > easily claim names on PyPI, and anyone who installs that package will
> > submit their information to that site. A couple that are up at the
> > moment:
> >
> > https://pypi.python.org/pypi/requirements-txt/1.1.1
> > https://pypi.python.org/pypi/ztz/0.1.1
> >
> > Do we delete them? Do we try to detect similar packages being uploaded
> > and block them? I suspect it's a waste of time to try to prevent this in
> > general, but maybe it's worth protecting likely names that people might
> > 'pip install' by mistake, such as requirements-txt.
> >
> > Thomas
> > ___
> > Distutils-SIG maillist  -  Distutils-SIG@python.org
> > https://mail.python.org/mailman/listinfo/distutils-sig
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] setuptools 33.1.1 - 35 issue with source package install from pypi

[Matt Joyce]

https://stackoverflow.com/questions/44120045/something-is-breaking-my-package-deployment

has full info on what i am seeing, but it suffices to say:

I use setuptools in my setup.py
I iterate a requirements.txt file to satisfy dependencies.
I publish as sdist to pypi

my package used to work fine as it currently is on pypi.  however as of
late i've been seeing the python setup.py invocation ( from pip install
python-symphony ) fail with a couple different failure scenarios.

one is it attempting to install what looks like a binary wheel ( that
doesn't exist on pypi or anywhere ) that ends up being a bunch of blank
stubs.   the other is a situation where the package looks 100% correct in
site-packages, but none of the module components are registered when you
try to use the imported module.

pip install --isolated --no-cache-dir python-symphony seems to avoid the
issue.  but it also tends to do weird stuff like break bpython by messing
up the pygments install.

curious if anyone was familiar with some sort of bug that i am stumbling
into here.

-Matt
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig