Re: Documentation on running Warehouse in your own production evironment?

2018-12-21 Thread Sumana Harihareswara
Hi and thanks for writing! And thanks for being clear and comprehensive about 
what you are looking for.

I'm sorry you didn't come across 
https://warehouse.readthedocs.io/application/#usage-assumptions-and-concepts -- 
maybe we need to flag that better. As it mentions:

> Warehouse is specifically the codebase for the official Python Package Index, 
> and thus focuses on architecture and features for PyPI and Test PyPI. People 
> and groups who want to run their own package indexes usually use other tools, 
> like devpi https://pypi.org/project/devpi-server/ .

You might also consider https://github.com/pypiserver/pypiserver or one of the 
other similar projects: 
https://github.com/pypiserver/pypiserver#similar-projects

I hope this helps!

(Sorry, I originally (yesterday) sent this off-list by mistake.)

-- 
Sumana Harihareswara
Changeset Consulting
https://changeset.nyc

On 12/20/18 9:23 AM, Christoph Bischko wrote:
> Hi all,
> 
> currently, the only official warehouse documentation at 
> https://warehouse.readthedocs.io seems to be quite sparse. Also it seems to 
> be aimed at developers, not end users. A lot about features and the hows of 
> setting up a warehouse instance is left in the dark.
> 
> For instance, there is the environment file 
> (https://github.com/pypa/warehouse/blob/master/dev/environment) that 
> contains some very necessary settings. I did not find out about it in the 
> documentation but by a painful reading of the Makefile. 
> Warehouse ships with "example data". I.e. on installation there are >40k 
> users and >30k dummy packages in the database by default - no instructions 
> on how to remove them. Again, I read the Makefile and altered the 
> example.sql database as a result.
> HTTPS seems to be disabled by default, with no documentation on how to set 
> it up properly and securely. 
> Account verification Mails, do not work out of the box - again no docs on 
> setting that up.
> Pip installation of packages on a client via the index on my local 
> Warehouse failed, because the links pointed to local host instead of the 
> actual fileserver. Again, the responsible setting FILES_BACKEND in the 
> environment is nowhere to be read about. 
> There is no information on whether it is possible and how, to set up 
> caching of the index at pypi.org with a local warehouse.
> 
> As you can see, I was able to resolve some of these issues with some 
> digging, reading of the code, trial and error and a bit pain. But my 
> questions are:
> 
>  - Am I missing something here? Is warehouse not meant to be used in 
> setting up your own local package index, i.e. for a company or educational 
> facility?
>   - Is there additional documentation I'm not aware of, something aimed at 
> system administrators and end-users that want to setup their own pypi, 
> because searching the web gives precious little and nothing usable?
>   - Are there (maybe 3rd-party) example configurations and guides for 
> warehouse?
> 
> The goal for a local instance of warehouse would be:
>  - Isolation of the local network from the internet (i.e. caching of pypi)
>  - Speedup of package installation via local network
>  - Having private packages locally that are not uploaded to pypi
> 
> I hope you can help me, and maybe documenting warehouse for end users will 
> get a higher priority as a result.
> Thanks,
> Christoph
> 
> 
> 



Documentation on running Warehouse in your own production evironment?

2018-12-20 Thread Christoph Bischko
Hi all,

currently, the only official warehouse documentation at 
https://warehouse.readthedocs.io seems to be quite sparse. Also it seems to 
be aimed at developers, not end users. A lot about features and the hows of 
setting up a warehouse instance is left in the dark.

For instance, there is the environment file 
(https://github.com/pypa/warehouse/blob/master/dev/environment) that 
contains some very necessary settings. I did not find out about it in the 
documentation but by a painful reading of the Makefile. 
Warehouse ships with "example data". I.e. on installation there are >40k 
users and >30k dummy packages in the database by default - no instructions 
on how to remove them. Again, I read the Makefile and altered the 
example.sql database as a result.
HTTPS seems to be disabled by default, with no documentation on how to set 
it up properly and securely. 
Account verification Mails, do not work out of the box - again no docs on 
setting that up.
Pip installation of packages on a client via the index on my local 
Warehouse failed, because the links pointed to local host instead of the 
actual fileserver. Again, the responsible setting FILES_BACKEND in the 
environment is nowhere to be read about. 
There is no information on whether it is possible and how, to set up 
caching of the index at pypi.org with a local warehouse.

As you can see, I was able to resolve some of these issues with some 
digging, reading of the code, trial and error and a bit pain. But my 
questions are:

 - Am I missing something here? Is warehouse not meant to be used in 
setting up your own local package index, i.e. for a company or educational 
facility?
  - Is there additional documentation I'm not aware of, something aimed at 
system administrators and end-users that want to setup their own pypi, 
because searching the web gives precious little and nothing usable?
  - Are there (maybe 3rd-party) example configurations and guides for 
warehouse?

The goal for a local instance of warehouse would be:
 - Isolation of the local network from the internet (i.e. caching of pypi)
 - Speedup of package installation via local network
 - Having private packages locally that are not uploaded to pypi

I hope you can help me, and maybe documenting warehouse for end users will 
get a higher priority as a result.
Thanks,
Christoph