Re: Figuring out library holes in the package installation process

[Brett Cannon]

Based on no one speaking up, I'm assuming I wasn't wrong with my outline. 
;) That looks like there may be one PEP to write and a few things to add to 
'packaging' (or other projects as appropriate).

On Monday, 19 March 2018 19:39:43 UTC-7, Brett Cannon wrote:
>
> With the big push we are all working towards standards, I thought it would 
> be interesting to outline the steps it takes to go from requesting a 
> package to be installed to it ending up on disk (see my first email on 
> such an outline 
> ).
>  
> After getting feedback on that outline, I then decided to try and pair each 
> complicated step of that outline with the PEP and a library that 
> essentially implements that step. I'm doing this to see where there might 
> be a missing PEP and where that are missing library implementations of a 
> specific PEP.
>
> I've ended up with some holes on both the PEP and library side, so I'm 
> hoping that either I'm overlooking something or that I've at least 
> identified a place where some help might be appreciated by people.
>
>
>1. Specify package requirement (PEP 508 
>/packaging 
>)
>   1. Translate name to PyPI-compatible name
>   2. Tease out requirement details (e.g. version, markers, etc.)
>2. Check if package is already installed (PEP 376 
>/XXX)
>3. Check wheel cache (XXX/XXX)
>4. Check PyPI for package
>5. Choose appropriate file
>   1. Get list of files (PEP 503 
>   /_)
>   2. Calculate best-fitting wheel (PEPs 425 
>    & 427 
>   /XXX)
>   3. Fallback to .tar.gz sdist if necessary & available
>6. Download file
>7. If sdist:
>   1. Extract
>   2. Read pyproject.toml (PEP 518/XXX)
>   3. Create venv
>   4. Install build dependencies
>  1. Static (PEP 518 /_)
>  2. Dynamic (PEP 517 
>  /XXX)
>   5. Build wheel (PEP 517 
>   /XXX)
>8. Cache wheel (XXX/XXX)
>9. Get dependency requirements of the wheel (PEP 566 
>/XXX)
>10. Check if dependencies are already installed (like step 2)
>11. Install missing dependencies (like step 12 below)
>12. Install wheel (PEP 427 
>/XXX)
>   1. To appropriate location (e.g. user, venv, etc.)
>   2. Library, scripts, etc. to appropriate locations
>13. Record the installation (PEP 427 
>/XXX)
>
>

Figuring out library holes in the package installation process

[Brett Cannon]

With the big push we are all working towards standards, I thought it would 
be interesting to outline the steps it takes to go from requesting a 
package to be installed to it ending up on disk (see my first email on such 
an outline 
).
 
After getting feedback on that outline, I then decided to try and pair each 
complicated step of that outline with the PEP and a library that 
essentially implements that step. I'm doing this to see where there might 
be a missing PEP and where that are missing library implementations of a 
specific PEP.

I've ended up with some holes on both the PEP and library side, so I'm 
hoping that either I'm overlooking something or that I've at least 
identified a place where some help might be appreciated by people.


   1. Specify package requirement (PEP 508 
   /packaging 
   )
  1. Translate name to PyPI-compatible name
  2. Tease out requirement details (e.g. version, markers, etc.)
   2. Check if package is already installed (PEP 376 
   /XXX)
   3. Check wheel cache (XXX/XXX)
   4. Check PyPI for package
   5. Choose appropriate file
  1. Get list of files (PEP 503 
  /_)
  2. Calculate best-fitting wheel (PEPs 425 
   & 427 
  /XXX)
  3. Fallback to .tar.gz sdist if necessary & available
   6. Download file
   7. If sdist:
  1. Extract
  2. Read pyproject.toml (PEP 518/XXX)
  3. Create venv
  4. Install build dependencies
 1. Static (PEP 518 /_)
 2. Dynamic (PEP 517 
 /XXX)
  5. Build wheel (PEP 517 
  /XXX)
   8. Cache wheel (XXX/XXX)
   9. Get dependency requirements of the wheel (PEP 566 
   /XXX)
   10. Check if dependencies are already installed (like step 2)
   11. Install missing dependencies (like step 12 below)
   12. Install wheel (PEP 427 
   /XXX)
  1. To appropriate location (e.g. user, venv, etc.)
  2. Library, scripts, etc. to appropriate locations
   13. Record the installation (PEP 427 
   /XXX)