Re: [pgAdmin4][Patch] Remake Docker container packaging

[Максим Кольцов]

2018-04-04 18:22 GMT+03:00 Dave Page <dp...@pgadmin.org>:
> Hi
>
> On Wed, Apr 4, 2018 at 2:46 PM, Максим Кольцов <kolma...@gmail.com> wrote:
>>
>>
>> So, I've fixed it. Patch attached. Now whole requirements.txt is
>> installed in one step, with no left-overs and incompabilities.
>
>
> I've committed the patch with some minor changes to avoid breaking
> compatibility with the older version:
>
> - The default ports are 80/443 again.
> - Username/password are set with PGADMIN_DEFAULT_EMAIL and
> PGADMIN_DEFAULT_PASSWORD (though, I have left them without defaults).
>
> Thanks!

Great, thank you very much!
Will you update image at docker hub?

> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company

Re: [pgAdmin4][Patch] Remake Docker container packaging

[Максим Кольцов]

2018-04-04 14:18 GMT+03:00 Dave Page <dp...@pgadmin.org>:
>
>
> On Wed, Apr 4, 2018 at 12:16 PM, Максим Кольцов <kolma...@gmail.com> wrote:
>>
>> 2018-04-04 13:55 GMT+03:00 Dave Page <dp...@pgadmin.org>:
>> > Hi
>> >
>> > On Wed, Apr 4, 2018 at 9:46 AM, Максим Кольцов <kolma...@gmail.com>
>> > wrote:
>> >>
>> >> I've updated patch. Now I drop tests and regressions.
>> >>
>> >> And I added new env var: PGADMIN_LISTEN_PORT. Default value is 8080 for
>> >> http and 8443 for https mode.
>> >>
>> >> Please review.
>> >
>> >
>> > I'm seeing the following error when building. It looks like it's caused
>> > by
>> > the new requirement on pycryptodome:
>>
>> Is this new requirement merged in master? I will rebase my patch.
>
>
> Yes it is.
>
>>
>>
>> In future, when chaning C-extensions in requirements.txt, it's needed
>> to update Dockerfile, line 43:
>> pip install --no-cache-dir psycopg2 pycrypto && \
>
>
> Ah, OK.
>
>>
>>
>> I install build deps, build C-extensions and remove build deps in one
>> RUN step in order to avoid having layer with build deps in image
>> stack.
>
>
> Makes sense.
>
>>
>> In fact, I can merge this with installation from requirements.txt.
>> There were some reasons I made it this way in the first place, but
>> it's not so relevant now.
>>
>> Will post patch later.
>
>
> Thanks!

So, I've fixed it. Patch attached. Now whole requirements.txt is
installed in one step, with no left-overs and incompabilities.

>>
>>
>> >   Running setup.py install for pycryptodome: started
>> > Running setup.py install for pycryptodome: finished with status
>> > 'error'
>> > Complete output from command /usr/local/bin/python -u -c "import
>> > setuptools,
>> >
>> > tokenize;__file__='/tmp/pip-build-mfrhme1c/pycryptodome/setup.py';f=getattr(tokenize,
>> > 'open', open)(__file__);code=f.read().replace('\r\n',
>> > '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
>> > /tmp/pip-4wlgoy0e-record/install-record.txt
>> > --single-version-externally-managed --compile:
>> > Testing support for x86intrin.h header
>> > Target does not support x86intrin.h header
>> > running install
>> > running build
>> > running build_py
>> > creating build/lib.linux-x86_64-3.6
>> > creating build/lib.linux-x86_64-3.6/Crypto
>> > copying lib/Crypto/__init__.py -> build/lib.linux-x86_64-3.6/Crypto
>> > creating build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/PKCS1_v1_5.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/_mode_ofb.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/ARC2.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/CAST.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/_mode_openpgp.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/ChaCha20.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/_mode_ccm.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/_mode_cbc.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/_mode_gcm.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/__init__.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/Salsa20.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/Blowfish.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/DES3.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/_mode_ctr.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/ARC4.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/PKCS1_OAEP.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Cipher
>> > copying lib/Crypto/Cipher/_mode_siv.py ->
>> > build/lib.linux-x86_64-3.6/Crypto/Ciphe

Re: [pgAdmin4][Patch] Remake Docker container packaging

[Максим Кольцов]

2018-04-04 14:16 GMT+03:00 Максим Кольцов <kolma...@gmail.com>:
> 2018-04-04 13:55 GMT+03:00 Dave Page <dp...@pgadmin.org>:
>> Hi
>>
>> On Wed, Apr 4, 2018 at 9:46 AM, Максим Кольцов <kolma...@gmail.com> wrote:
>>>
>>> I've updated patch. Now I drop tests and regressions.
>>>
>>> And I added new env var: PGADMIN_LISTEN_PORT. Default value is 8080 for
>>> http and 8443 for https mode.
>>>
>>> Please review.
>>
>>
>> I'm seeing the following error when building. It looks like it's caused by
>> the new requirement on pycryptodome:
>
> Is this new requirement merged in master? I will rebase my patch.
>
> In future, when chaning C-extensions in requirements.txt, it's needed
> to update Dockerfile, line 43:
> pip install --no-cache-dir psycopg2 pycrypto && \
>
> I install build deps, build C-extensions and remove build deps in one
> RUN step in order to avoid having layer with build deps in image
> stack.
> In fact, I can merge this with installation from requirements.txt.
> There were some reasons I made it this way in the first place, but
> it's not so relevant now.
>
> Will post patch later.

Also, please note line 26 in Dockerfile:

RUN pip install --no-cache-dir \
sphinx flask_babel flask_security flask_paranoid python-dateutil
flask_sqlalchemy \
flask_gravatar simplejson

Here I install packages that are imported during documentation build,
to avoid installing whole load of heavy packages and C-extensions in
docs-only intermediate container.
So, when these deps are changed, Dockerfile should be updated as well.

>>   Running setup.py install for pycryptodome: started
>> Running setup.py install for pycryptodome: finished with status 'error'
>> Complete output from command /usr/local/bin/python -u -c "import
>> setuptools,
>> tokenize;__file__='/tmp/pip-build-mfrhme1c/pycryptodome/setup.py';f=getattr(tokenize,
>> 'open', open)(__file__);code=f.read().replace('\r\n',
>> '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
>> /tmp/pip-4wlgoy0e-record/install-record.txt
>> --single-version-externally-managed --compile:
>> Testing support for x86intrin.h header
>> Target does not support x86intrin.h header
>> running install
>> running build
>> running build_py
>> creating build/lib.linux-x86_64-3.6
>> creating build/lib.linux-x86_64-3.6/Crypto
>> copying lib/Crypto/__init__.py -> build/lib.linux-x86_64-3.6/Crypto
>> creating build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/PKCS1_v1_5.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_ofb.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/ARC2.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/CAST.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_openpgp.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/ChaCha20.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_ccm.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_cbc.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_gcm.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/__init__.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/Salsa20.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/Blowfish.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/DES3.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_ctr.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/ARC4.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/PKCS1_OAEP.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_siv.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/AES.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_eax.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_ecb.py ->
>> build/lib.linux-x86_64-3.6/Crypto/Cipher
>> copying lib/Crypto/Cipher/_mode_ocb.py ->
>> build/

Re: [pgAdmin4][Patch] Remake Docker container packaging

[Максим Кольцов]

2018-04-04 13:55 GMT+03:00 Dave Page <dp...@pgadmin.org>:
> Hi
>
> On Wed, Apr 4, 2018 at 9:46 AM, Максим Кольцов <kolma...@gmail.com> wrote:
>>
>> I've updated patch. Now I drop tests and regressions.
>>
>> And I added new env var: PGADMIN_LISTEN_PORT. Default value is 8080 for
>> http and 8443 for https mode.
>>
>> Please review.
>
>
> I'm seeing the following error when building. It looks like it's caused by
> the new requirement on pycryptodome:

Is this new requirement merged in master? I will rebase my patch.

In future, when chaning C-extensions in requirements.txt, it's needed
to update Dockerfile, line 43:
pip install --no-cache-dir psycopg2 pycrypto && \

I install build deps, build C-extensions and remove build deps in one
RUN step in order to avoid having layer with build deps in image
stack.
In fact, I can merge this with installation from requirements.txt.
There were some reasons I made it this way in the first place, but
it's not so relevant now.

Will post patch later.

>   Running setup.py install for pycryptodome: started
> Running setup.py install for pycryptodome: finished with status 'error'
> Complete output from command /usr/local/bin/python -u -c "import
> setuptools,
> tokenize;__file__='/tmp/pip-build-mfrhme1c/pycryptodome/setup.py';f=getattr(tokenize,
> 'open', open)(__file__);code=f.read().replace('\r\n',
> '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
> /tmp/pip-4wlgoy0e-record/install-record.txt
> --single-version-externally-managed --compile:
> Testing support for x86intrin.h header
> Target does not support x86intrin.h header
> running install
> running build
> running build_py
> creating build/lib.linux-x86_64-3.6
> creating build/lib.linux-x86_64-3.6/Crypto
> copying lib/Crypto/__init__.py -> build/lib.linux-x86_64-3.6/Crypto
> creating build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/PKCS1_v1_5.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_ofb.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/ARC2.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/CAST.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_openpgp.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/ChaCha20.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_ccm.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_cbc.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_gcm.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/__init__.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/Salsa20.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/Blowfish.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/DES3.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_ctr.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/ARC4.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/PKCS1_OAEP.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_siv.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/AES.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_eax.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_ecb.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_ocb.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/DES.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> copying lib/Crypto/Cipher/_mode_cfb.py ->
> build/lib.linux-x86_64-3.6/Crypto/Cipher
> creating build/lib.linux-x86_64-3.6/Crypto/Hash
> copying lib/Crypto/Hash/SHA256.py ->
> build/lib.linux-x86_64-3.6/Crypto/Hash
> copying lib/Crypto/Hash/SHAKE256.py ->
> build/lib.linux-x86_64-3.6/Crypto/Hash
> copying lib/Crypto/Hash/SHA384.py ->
> build/lib.linux-x86_64-3.6/Crypto/Hash
> copying lib/Crypto/Hash/SHA224.py ->
> build/lib.linux-x86_64-3.6/Crypto/Hash
> copying lib/Crypto/Hash/__init__.py ->
> build/lib.linux-x86_64-3.6/Crypto/Hash
> copying lib/Crypto/Hash/SHAKE128.py ->
> build/lib.linux-x86_64-3.6/Crypto/Hash
> copying lib/Crypto/Hash/HMAC.py

Re: [pgAdmin4][Patch] Remake Docker container packaging

[Максим Кольцов]

пн, 2 апр. 2018 г., 7:54 Murtuza Zabuawala <
murtuza.zabuaw...@enterprisedb.com>:

> Hello,
>
> I tested the patch and it is working fine.
>
> - I had clean system before but after running DockerBuild, I can see 3
> different images of pgAdmin4, Is this expected?
>
Yes, this is expected and is the same for old docker image. Two unnamed
images are temporary images from build process and can be safely removed,
if you don't want cache for next builds.


> - We can remove /tests/ folder from each folders, we can also remove
> /web/regression/ folder from the container, they are used to run tests.
> We don't require them on production code.
>
Sounds reasonable. In fact, I believe I did that when making a prototype,
but forgot to replicate in this patch. Will do.

- Can we make the ports configurable?
> https://redmine.postgresql.org/issues/2997
>
This is certainly doable. Please note, by the way, that pgadmin is now
started on port 8080 or 8443 by default.

- Allow user to choose whether to run it as Single user mode(SERVER_MODE =
> False) or Multiuser mode(SERVER_MODE = True)
>
>
> --
> Regards,
> Murtuza Zabuawala
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
>
> On Sat, Mar 31, 2018 at 11:19 PM, Максим Кольцов <kolma...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I've been discussing this with Dave for about a month now. Today I
>> finally present a proposed patch to update pgadmin4's Docker
>> packaging.
>>
>> Key features of this update:
>> - Main image is based on python:3.6-alpine3.7.
>>   Using Alpine linux leads to much smaller image
>> - All build is done with Docker multi-stage build. First of all build
>> the frontend in node:6 image,
>>   then build Sphinx documentation in separate Python container and in
>> the end just install all
>>   dependencies in a clean python:3.6-alpine3.7 image, so that it does
>> not have any leftovers from the build
>>   process and we don't rely on any tools available on the host.
>> - Use Gunicorn (http://gunicorn.org) as lightweight HTTP / WSGI server.
>>   Gunicorn supports both HTTP and HTTPS.
>> - Install Alpine postgresql-client package, which includes pg_dump and
>> other tools and config
>>   PgAdmin to find these tools by default
>> - Byte-compile all PgAdmin Python code in Dockerfile with optimization
>> (-O) enabled. This way Python
>>does not have to compile modules on each container restart and
>> consume space in overlay fs
>>
>> Please find attached patch from "git format-patch".
>>
>
>

[pgAdmin4][Patch] Remake Docker container packaging

[Максим Кольцов]

Hi all,

I've been discussing this with Dave for about a month now. Today I
finally present a proposed patch to update pgadmin4's Docker
packaging.

Key features of this update:
- Main image is based on python:3.6-alpine3.7.
  Using Alpine linux leads to much smaller image
- All build is done with Docker multi-stage build. First of all build
the frontend in node:6 image,
  then build Sphinx documentation in separate Python container and in
the end just install all
  dependencies in a clean python:3.6-alpine3.7 image, so that it does
not have any leftovers from the build
  process and we don't rely on any tools available on the host.
- Use Gunicorn (http://gunicorn.org) as lightweight HTTP / WSGI server.
  Gunicorn supports both HTTP and HTTPS.
- Install Alpine postgresql-client package, which includes pg_dump and
other tools and config
  PgAdmin to find these tools by default
- Byte-compile all PgAdmin Python code in Dockerfile with optimization
(-O) enabled. This way Python
   does not have to compile modules on each container restart and
consume space in overlay fs

Please find attached patch from "git format-patch".
From 67e387525a7c832958858c7ec1a1b7076382090e Mon Sep 17 00:00:00 2001
From: Maxim Koltsov 
Date: Sat, 31 Mar 2018 20:37:51 +0300
Subject: [PATCH] Re-make Docker container packaging

Key features of this update:
- Main image is based on python:3.6-alpine3.7.
  Using Alpine linux leads to much smaller image
- All build is done with Docker multi-stage build. First of all build the frontend in node:6 image,
  then build Sphinx documentation in separate Python container and in the end just install all
  dependencies in a clean python:3.6-alpine3.7 image, so that it does not have any leftovers from the build
  process and we don't rely on any tools available on the host.
- Use Gunicorn (http://gunicorn.org) as lightweight HTTP / WSGI server.
  Gunicorn supports both HTTP and HTTPS.
- Install Alpine postgresql-client package, which includes pg_dump and other tools and config
  PgAdmin to find these tools by default
- Byte-compile all PgAdmin Python code in Dockerfile with optimization (-O) enabled. This way Python
  does not have to compile modules on each container restart and consume space in overlay fs
---
 pkg/docker/Dockerfile   | 89 -
 pkg/docker/README   | 58 ++---
 pkg/docker/build.sh | 57 +
 pkg/docker/config_distro.py |  4 ++
 pkg/docker/entry.sh | 29 ---
 pkg/docker/entrypoint.sh| 21 +++
 pkg/docker/pgadmin4.conf.j2 | 43 --
 pkg/docker/run_pgadmin.py   |  4 ++
 8 files changed, 112 insertions(+), 193 deletions(-)
 create mode 100644 pkg/docker/config_distro.py
 delete mode 100644 pkg/docker/entry.sh
 create mode 100755 pkg/docker/entrypoint.sh
 delete mode 100644 pkg/docker/pgadmin4.conf.j2
 create mode 100644 pkg/docker/run_pgadmin.py

diff --git a/pkg/docker/Dockerfile b/pkg/docker/Dockerfile
index 1c1dde27..083dbd60 100644
--- a/pkg/docker/Dockerfile
+++ b/pkg/docker/Dockerfile
@@ -7,58 +7,63 @@
 #
 #
 
-# Get the basics out of the way
-FROM centos:latest
+# First of all, build frontend with NodeJS in a separate builder container
+# Node-6 with ABI v48 is supported by all needed C++ packages
+FROM node:6 AS node-builder
 
-LABEL name="pgAdmin 4" \
-vendor="The pgAdmin Development Team" \
-license="PostgreSQL"
+COPY ./pgadmin4/web/ /pgadmin4/web/
+WORKDIR /pgadmin4/web
 
-# We only need the web/ directory, and a few other things
-COPY web /var/www/pgadmin
-COPY requirements.txt /var/www/pgadmin
+RUN yarn install --cache-folder ./ycache --verbose && \
+yarn run bundle && \
+rm -rf ./ycache ./pgadmin/static/js/generated/.cache
 
-# Install everything we need. Use easy_install to get pip, to avoid setting up EPEL
-RUN yum install -y python-setuptools python-devel httpd mod_wsgi mod_ssl gcc
-RUN easy_install pip
-RUN pip install j2cli
+# Build Sphinx documentation in separate container
+FROM python:3.6-alpine3.7 as docs-builder
 
-# Now install the Python runtime dependencies
-RUN pip install -r /var/www/pgadmin/requirements.txt
+# Install only dependencies absolutely required for documentation building
+RUN apk add --no-cache make
+RUN pip install --no-cache-dir \
+sphinx flask_babel flask_security flask_paranoid python-dateutil flask_sqlalchemy \
+flask_gravatar simplejson
 
-# Create required directories for config
+COPY ./pgadmin4/ /pgadmin4
 
+RUN LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 make -C /pgadmin4/docs/en_US -f Makefile.sphinx html
 
-# Create required directories for running
-RUN mkdir -p /var/log/pgadmin
-RUN chown -R apache /var/log/pgadmin
-RUN mkdir -p /var/lib/pgadmin
-RUN chown -R apache /var/lib/pgadmin
-RUN mkdir -p /certs
-RUN chown -R apache /certs
-RUN chmod 700 /certs
+# Then install 

Re: Proposal for changes in official Docker image

[Максим Кольцов]

2018-03-19 17:55 GMT+03:00 Dave Page <dp...@pgadmin.org>:
> Hi
>
> On Mon, Mar 19, 2018 at 11:19 AM, Dave Page <dp...@pgadmin.org> wrote:
>>
>> Hi
>>
>> On Fri, Mar 16, 2018 at 8:52 PM, Максим Кольцов <kolma...@gmail.com>
>> wrote:
>>>
>>>
>>> I've switched to Gunicorn, adding SSL support. It has the same
>>> interface as the original container: PGADMIN_ENABLE_TLS,
>>> /certs/server.key and /certs/server.cert.
>>> I also incorporated building of sphinx manual in Dockerfile, so now
>>> the image should be complete.
>>
>>
>> Nice!
>>
>>>
>>>
>>> I noticed that I can't use gunicorn forking worker with pgadmin4, this
>>> is probably caused by session implementation, but I'm not sure. You
>>> can investigate this by using e.g. `-w 4` in entrypoint.sh, otherwise
>>> it's working fine with single-process threaded worker.
>>
>>
>> This is expected, and the reason why is explained here:
>> https://www.pgadmin.org/docs/pgadmin4/2.x/server_deployment.html#requirements
>>
>>>
>>>
>>> I will make my work into a patch and send it to the mail list soon.
>>> Meanwhile, it'd be great if you tested the updated image at
>>> https://hub.docker.com/r/maksbotan/pgadmin4/
>>
>>
>> I'll try to take a look later in the week (busy prepping for a release at
>> the moment).
>
>
> I actually found some time for this today. It's looking pretty good - TLS
> vs. non-TLS seems to work well. The couple of things I did notice were:
>
> - PGADMIN_SERVER_NAME doesn't appear to be supported. This was added at user
> request, for security reasons (to help ensure the connection is going where
> expected). I'm not entirely convinced of the value of that, but if it's
> fairly painless to add, it may well be worth it.

Can you explain the meaning of this option, or maybe give me a link to
original feature request?
As far as I understand, Apache uses this to identify virtual hosts
based on HTTP Host header,
but there are no virtual hosts in Gunicorn, so no need for this.

> - If you don't map /var/lib/pgadmin (in my case, to a directory already
> containing a config), then I was seeing the following error. I would expect
> it to init within the container if the config directory isn't mapped:
>
> piranha:web dpage$ docker logs affectionate_spence
> sh: -z: unknown operand
> NOTE: Configuring authentication for SERVER mode.
>
> Enter the email address and password to use for the initial pgAdmin user
> account:
>
> Traceback (most recent call last):
>   File "run_pgadmin.py", line 4, in 
> from pgAdmin4 import app
>   File "/pgadmin4/pgAdmin4.py", line 67, in 
> app = create_app()
>   File "/pgadmin4/pgadmin/__init__.py", line 306, in create_app
> db_upgrade(app)
>   File "/pgadmin4/pgadmin/setup/db_upgrade.py", line 25, in db_upgrade
> flask_migrate.upgrade(migration_folder)
>   File "/usr/local/lib/python3.6/site-packages/flask_migrate/__init__.py",
> line 244, in upgrade
> command.upgrade(config, revision, sql=sql, tag=tag)
>   File "/usr/local/lib/python3.6/site-packages/alembic/command.py", line
> 254, in upgrade
> script.run_env()
>   File "/usr/local/lib/python3.6/site-packages/alembic/script/base.py", line
> 427, in run_env
> util.load_python_file(self.dir, 'env.py')
>   File "/usr/local/lib/python3.6/site-packages/alembic/util/pyfiles.py",
> line 81, in load_python_file
> module = load_module_py(module_id, path)
>   File "/usr/local/lib/python3.6/site-packages/alembic/util/compat.py", line
> 83, in load_module_py
> spec.loader.exec_module(module)
>   File "/pgadmin4/pgadmin/setup/../../migrations/env.py", line 94, in
> 
> run_migrations_online()
>   File "/pgadmin4/pgadmin/setup/../../migrations/env.py", line 87, in
> run_migrations_online
> context.run_migrations()
>   File "", line 8, in run_migrations
>   File
> "/usr/local/lib/python3.6/site-packages/alembic/runtime/environment.py",
> line 836, in run_migrations
> self.get_context().run_migrations(**kw)
>   File
> "/usr/local/lib/python3.6/site-packages/alembic/runtime/migration.py", line
> 330, in run_migrations
> step.migration_fn(**kw)
>   File "/pgadmin4/migrations/versions/fdc58d9bd449_.py", line 112, in
> upgrade
> email, password = user_info()
>   File "/pgadmin4/pgadmin/setup/user_info.py", line 55, in user_info
> email = input("Email address: ")
> EOFError: EOF when rea

Re: Proposal for changes in official Docker image

[Максим Кольцов]

2018-02-26 13:46 GMT+03:00 Dave Page <dp...@pgadmin.org>:
> Hi
>
> On Mon, Feb 26, 2018 at 10:09 AM, Максим Кольцов <kolma...@gmail.com> wrote:
>>
>> 2018-02-25 20:59 GMT+03:00 Dave Page <dp...@pgadmin.org>:
>> > Hi
>> >
>> > On Sat, Feb 24, 2018 at 9:04 PM, Максим Кольцов <kolma...@gmail.com>
>> > wrote:
>> >>
>> >> Hi
>> >>
>> >> 2018-02-19 12:13 GMT+03:00 Dave Page <dp...@pgadmin.org>:
>> >> > Hi
>> >> >
>> >> > On Sun, Feb 18, 2018 at 5:41 PM, Максим Кольцов <kolma...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi!
>> >> >>
>> >> >> I accidentially sent this email to pgsql-hackers yesterday, sorry!
>> >> >>
>> >> >> First of all, thanks for the great app :)
>> >> >>
>> >> >> I started using PgAdmin with docker image (dpage/pgadmin4) a few
>> >> >> weeks
>> >> >> ago, however I thought that it had some issues, so I decided to make
>> >> >> my own image. Some of the advantages:
>> >> >>
>> >> >> - Use alpine linux instead of centos to greatly reduce image size
>> >> >> (170MB vs 560MB)
>> >> >> - Use lightweight pure-python HTTP server waitress instead of heavy
>> >> >> apache/mod_wsgi
>> >> >> - Use python 3.6
>> >> >>
>> >> >> You can test the image at
>> >> >> https://hub.docker.com/r/maksbotan/pgadmin4/
>> >> >> Readme contains more detailed explanation and usage instructions.
>> >> >>
>> >> >> The Dockerfile is hosted at github:
>> >> >> https://github.com/maksbotan/pgadmin4_docker
>> >> >>
>> >> >> If you find my work useful, I'd love to make a contribution with
>> >> >> these
>> >> >> scripts, after some discussion with pgadmin developers and further
>> >> >> improvements.
>> >> >
>> >> >
>> >> > Please feel free to submit patches to the existing code. I have no
>> >> > objection
>> >> > to the any of the alternate design decisions you've made (in
>> >> > principal),
>> >> > except for the intentional lack of SSL support.
>> >> >
>> >> > Thanks, Dave.
>> >>
>> >> I updated my image to simplify installing of Python packages. I
>> >> decided I do not need a separate build step after all.
>> >> Can you point me at documentation on submitting patches to pgadmin?
>> >
>> >
>> > There are some docs on the git repo and mailing list at
>> > https://www.pgadmin.org/development/resources/. To submit a patch, send
>> > an
>> > email to the hackers list describing the patch and attaching the "git
>> > diff"
>> > formatted patch file.
>> >
>> >>
>> >>
>> >> What are your points in including SSL support into container? This can
>> >> be done by using, for example, gunicorn instead of waitress,
>> >> but I believe that this should be handled by reverse-proxy, like
>> >> nginx, in production environment. In non-production environment, i.e.
>> >> on developer's localhost, you do not need SSL at all.
>> >>
>> >> By the way, in my opinion, on production there is one more task to be
>> >> handled by reverse-proxy - static files. By that I mean that all
>> >> static, not-changing files accessible at '/static/' URL should be
>> >> extracted from the container and served by nginx from a local folder.
>> >> This does not mean we shouldn't keep them in the image -- it's very
>> >> convenient for localhost usage. I haven't found a way to extract
>> >> all Flask's static files yet.
>> >
>> >
>> > Well that additional complexity is a very good reason why using two
>> > containers for this is overkill. Having two containers to run pgAdmin
>> > makes
>> > things unnecessarily complex in my opinion, especially given that it can
>> > (and is in the current container) achieved with the simple addition of a
>> > config snippet for Apache and mod_ssl. The current trend for micro
>> > services
>> > can easily be taken too far - we should keep the KISS principle in mind.
>>

Re: Proposal for changes in official Docker image

[Максим Кольцов]

2018-02-25 20:59 GMT+03:00 Dave Page <dp...@pgadmin.org>:
> Hi
>
> On Sat, Feb 24, 2018 at 9:04 PM, Максим Кольцов <kolma...@gmail.com> wrote:
>>
>> Hi
>>
>> 2018-02-19 12:13 GMT+03:00 Dave Page <dp...@pgadmin.org>:
>> > Hi
>> >
>> > On Sun, Feb 18, 2018 at 5:41 PM, Максим Кольцов <kolma...@gmail.com>
>> > wrote:
>> >>
>> >> Hi!
>> >>
>> >> I accidentially sent this email to pgsql-hackers yesterday, sorry!
>> >>
>> >> First of all, thanks for the great app :)
>> >>
>> >> I started using PgAdmin with docker image (dpage/pgadmin4) a few weeks
>> >> ago, however I thought that it had some issues, so I decided to make
>> >> my own image. Some of the advantages:
>> >>
>> >> - Use alpine linux instead of centos to greatly reduce image size
>> >> (170MB vs 560MB)
>> >> - Use lightweight pure-python HTTP server waitress instead of heavy
>> >> apache/mod_wsgi
>> >> - Use python 3.6
>> >>
>> >> You can test the image at https://hub.docker.com/r/maksbotan/pgadmin4/
>> >> Readme contains more detailed explanation and usage instructions.
>> >>
>> >> The Dockerfile is hosted at github:
>> >> https://github.com/maksbotan/pgadmin4_docker
>> >>
>> >> If you find my work useful, I'd love to make a contribution with these
>> >> scripts, after some discussion with pgadmin developers and further
>> >> improvements.
>> >
>> >
>> > Please feel free to submit patches to the existing code. I have no
>> > objection
>> > to the any of the alternate design decisions you've made (in principal),
>> > except for the intentional lack of SSL support.
>> >
>> > Thanks, Dave.
>>
>> I updated my image to simplify installing of Python packages. I
>> decided I do not need a separate build step after all.
>> Can you point me at documentation on submitting patches to pgadmin?
>
>
> There are some docs on the git repo and mailing list at
> https://www.pgadmin.org/development/resources/. To submit a patch, send an
> email to the hackers list describing the patch and attaching the "git diff"
> formatted patch file.
>
>>
>>
>> What are your points in including SSL support into container? This can
>> be done by using, for example, gunicorn instead of waitress,
>> but I believe that this should be handled by reverse-proxy, like
>> nginx, in production environment. In non-production environment, i.e.
>> on developer's localhost, you do not need SSL at all.
>>
>> By the way, in my opinion, on production there is one more task to be
>> handled by reverse-proxy - static files. By that I mean that all
>> static, not-changing files accessible at '/static/' URL should be
>> extracted from the container and served by nginx from a local folder.
>> This does not mean we shouldn't keep them in the image -- it's very
>> convenient for localhost usage. I haven't found a way to extract
>> all Flask's static files yet.
>
>
> Well that additional complexity is a very good reason why using two
> containers for this is overkill. Having two containers to run pgAdmin makes
> things unnecessarily complex in my opinion, especially given that it can
> (and is in the current container) achieved with the simple addition of a
> config snippet for Apache and mod_ssl. The current trend for micro services
> can easily be taken too far - we should keep the KISS principle in mind.

I did not mean to run two containers. I mean that pgadmin image, as I
picture it, may serve two purposes:

- localhost deployment on developer's machine to ease interaction with
postgres DB, local or remote.
  In this mode container serves it's own static files and is
accessible via plain HTTP
- Deployment in enterprise production environment, for many users,
possibly accessible from the Internet.
  In this mode container should only serve the API, possibly running
in several replicas. static files and SSL
  termination should be done by _existing_ nginx or something else
present in that organisation. For that I'd wish
  to have a way to extract static files from the container for
deployment, but not changing anything in the image.

> Another reason for including SSL support, is that users have asked for it.

In my humble opinion, if users want SSL support in application
container, they are doing something wrong and are
asking for troubles. But I respect this choice and I'm ready to allow
for it. I'll integrate gunicorn server in the image, which
supports SSL.

> Regards, Dave.
>
> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company

Re: Proposal for changes in official Docker image

[Максим Кольцов]

Hi

2018-02-19 12:13 GMT+03:00 Dave Page <dp...@pgadmin.org>:
> Hi
>
> On Sun, Feb 18, 2018 at 5:41 PM, Максим Кольцов <kolma...@gmail.com> wrote:
>>
>> Hi!
>>
>> I accidentially sent this email to pgsql-hackers yesterday, sorry!
>>
>> First of all, thanks for the great app :)
>>
>> I started using PgAdmin with docker image (dpage/pgadmin4) a few weeks
>> ago, however I thought that it had some issues, so I decided to make
>> my own image. Some of the advantages:
>>
>> - Use alpine linux instead of centos to greatly reduce image size
>> (170MB vs 560MB)
>> - Use lightweight pure-python HTTP server waitress instead of heavy
>> apache/mod_wsgi
>> - Use python 3.6
>>
>> You can test the image at https://hub.docker.com/r/maksbotan/pgadmin4/
>> Readme contains more detailed explanation and usage instructions.
>>
>> The Dockerfile is hosted at github:
>> https://github.com/maksbotan/pgadmin4_docker
>>
>> If you find my work useful, I'd love to make a contribution with these
>> scripts, after some discussion with pgadmin developers and further
>> improvements.
>
>
> Please feel free to submit patches to the existing code. I have no objection
> to the any of the alternate design decisions you've made (in principal),
> except for the intentional lack of SSL support.
>
> Thanks, Dave.

I updated my image to simplify installing of Python packages. I
decided I do not need a separate build step after all.
Can you point me at documentation on submitting patches to pgadmin?

What are your points in including SSL support into container? This can
be done by using, for example, gunicorn instead of waitress,
but I believe that this should be handled by reverse-proxy, like
nginx, in production environment. In non-production environment, i.e.
on developer's localhost, you do not need SSL at all.

By the way, in my opinion, on production there is one more task to be
handled by reverse-proxy - static files. By that I mean that all
static, not-changing files accessible at '/static/' URL should be
extracted from the container and served by nginx from a local folder.
This does not mean we shouldn't keep them in the image -- it's very
convenient for localhost usage. I haven't found a way to extract
all Flask's static files yet.

> --
> Dave Page
> Blog: http://pgsnake.blogspot.com
> Twitter: @pgsnake
>
> EnterpriseDB UK: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company

Proposal for changes in official Docker image

[Максим Кольцов]

Hi!

I accidentially sent this email to pgsql-hackers yesterday, sorry!

First of all, thanks for the great app :)

I started using PgAdmin with docker image (dpage/pgadmin4) a few weeks
ago, however I thought that it had some issues, so I decided to make
my own image. Some of the advantages:

- Use alpine linux instead of centos to greatly reduce image size
(170MB vs 560MB)
- Use lightweight pure-python HTTP server waitress instead of heavy
apache/mod_wsgi
- Use python 3.6

You can test the image at https://hub.docker.com/r/maksbotan/pgadmin4/
Readme contains more detailed explanation and usage instructions.

The Dockerfile is hosted at github: https://github.com/maksbotan/pgadmin4_docker

If you find my work useful, I'd love to make a contribution with these
scripts, after some discussion with pgadmin developers and further
improvements.

Looking forward for an answer,
Maxim.