rydka opened a new issue, #25182:
URL: https://github.com/apache/superset/issues/25182
Hello !
My we have been working with super for a few month now and yesterday a
shortage of disk forced us to restart the server.
While restarting with the docker-compose-non-dev.yml pulled the
superset:latest-dev and since then the superset_worker_beat fails to Create and
Initialize the database and exits.
In the process we want to keep the information present in the db.
Here is the docker-compose-non-dev.yml used to restart the containers
```
x-superset-image: &superset-image apache/superset:${TAG:-latest-dev}
x-superset-depends-on: &superset-depends-on
- db_test
- redis
x-superset-volumes: &superset-volumes
# /app/pythonpath_docker will be appended to the PYTHONPATH in the final
container
- ./docker:/app/docker
- superset_home:/app/superset_home
version: "3.7"
services:
redis:
image: redis:7
container_name: superset_cache
#restart: always
volumes:
- redis:/data
db_test:
env_file: docker/.env-non-dev
image: postgres:14
container_name: superset_db
#restart: always
volumes:
- db_home:/var/lib/postgresql/data
superset:
env_file: docker/.env-non-dev
image: *superset-image
container_name: superset_app
command: ["/app/docker/docker-bootstrap.sh", "app-gunicorn"]
user: "root"
restart: unless-stopped
ports:
- 8088:8088
depends_on: *superset-depends-on
volumes: *superset-volumes
superset-init:
image: *superset-image
container_name: superset_init
command: ["/app/docker/docker-init.sh"]
env_file: docker/.env-non-dev
depends_on: *superset-depends-on
user: "root"
volumes: *superset-volumes
healthcheck:
disable: true
superset-worker:
image: *superset-image
container_name: superset_worker
command: ["/app/docker/docker-bootstrap.sh", "worker"]
env_file: docker/.env-non-dev
restart: unless-stopped
depends_on: *superset-depends-on
user: "root"
volumes: *superset-volumes
healthcheck:
test: ["CMD-SHELL", "celery inspect ping -A
superset.tasks.celery_app:app -d celery@$$HOSTNAME"]
restart: always
superset-worker-beat:
image: *superset-image
container_name: superset_worker_beat
command: ["/app/docker/docker-bootstrap.sh", "beat"]
env_file: docker/.env-non-dev
restart: unless-stopped
depends_on: *superset-depends-on
user: "root"
volumes: *superset-volumes
healthcheck:
disable: true
restart: always
volumes:
superset_home:
external: false
db_home:
external: false
redis:
external: false
```
You can find the logs of the containers running below :
superset_app | Skipping local overrides
superset_app | Starting web app...
superset_cache | 1:C 05 Sep 2023 09:23:06.334 # oO0OoO0OoO0Oo Redis
is starting oO0OoO0OoO0Oo
superset_cache | 1:C 05 Sep 2023 09:23:06.334 # Redis
version=7.0.11, bits=64, commit=00000000, modified=0, pid=1, just started
superset_cache | 1:C 05 Sep 2023 09:23:06.334 # Warning: no config
file specified, using the default config. In order to specify a config file use
redis-server /path/to/redis.conf
superset_init | Skipping local overrides
superset_init |
superset_init |
######################################################################
superset_init |
superset_init |
superset_init | Init Step 1/4 [Starting] -- Applying DB migrations
superset_init |
superset_init |
superset_init |
######################################################################
superset_init |
superset_cache | 1:M 05 Sep 2023 09:23:06.334 * monotonic clock:
POSIX clock_gettime
superset_worker | Skipping local overrides
superset_worker | Starting Celery worker...
superset_db |
superset_db | PostgreSQL Database directory appears to contain a
database; Skipping initialization
superset_db |
superset_cache | 1:M 05 Sep 2023 09:23:06.335 * Running
mode=standalone, port=6379.
superset_cache | 1:M 05 Sep 2023 09:23:06.335 # Server initialized
superset_cache | 1:M 05 Sep 2023 09:23:06.335 # WARNING Memory
overcommit must be enabled! Without it, a background save or replication may
fail under low memory condition. Being disabled, it can can also cause failures
without low memory condition, see
https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add
'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the
command 'sysctl vm.overcommit_memory=1' for this to take effect.
superset_db | 2023-09-05 09:23:06.496 UTC [1] LOG: starting
PostgreSQL 14.9 (Debian 14.9-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by
gcc (Debian 12.2.0-14) 12.2.0, 64-bit
superset_cache | 1:M 05 Sep 2023 09:23:06.338 * Loading RDB
produced by version 7.0.11
superset_cache | 1:M 05 Sep 2023 09:23:06.338 * RDB age 955 seconds
superset_cache | 1:M 05 Sep 2023 09:23:06.338 * RDB memory usage
when created 0.95 Mb
superset_db | 2023-09-05 09:23:06.496 UTC [1] LOG: listening on
IPv4 address "0.0.0.0", port 5432
superset_db | 2023-09-05 09:23:06.496 UTC [1] LOG: listening on
IPv6 address "::", port 5432
superset_db | 2023-09-05 09:23:06.510 UTC [1] LOG: listening on
Unix socket "/var/run/postgresql/.s.PGSQL.5432"
superset_cache | 1:M 05 Sep 2023 09:23:06.341 * Done loading RDB,
keys loaded: 303, keys expired: 16.
superset_cache | 1:M 05 Sep 2023 09:23:06.341 * DB loaded from
disk: 0.005 seconds
superset_cache | 1:M 05 Sep 2023 09:23:06.341 * Ready to accept
connections
superset_db | 2023-09-05 09:23:06.672 UTC [27] LOG: database
system was shut down at 2023-09-05 09:07:11 UTC
superset_db | 2023-09-05 09:23:06.684 UTC [1] LOG: database
system is ready to accept connections
superset_worker_beat | Skipping local overrides
superset_worker_beat | Starting Celery beat...
superset_app | [2023-09-05 09:23:07 +0000] [8] [INFO] Starting
gunicorn 20.1.0
superset_app | [2023-09-05 09:23:07 +0000] [8] [INFO] Listening
at: http://0.0.0.0:8088 (8)
superset_app | [2023-09-05 09:23:07 +0000] [8] [INFO] Using
worker: gthread
superset_app | [2023-09-05 09:23:07 +0000] [10] [INFO] Booting
worker with pid: 10
superset_worker_beat | logging was configured successfully
superset_worker_beat | 2023-09-05
09:23:09,592:INFO:superset.utils.logging_configurator:logging was configured
successfully
superset_worker_beat | 2023-09-05 09:23:09,594:INFO:root:Configured event
logger of type <class 'superset.utils.log.DBEventLogger'>
superset_worker_beat | We haven't found any Content Security Policy (CSP)
defined in the configurations. Please make sure to configure CSP using the
TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software.
Failing to configure CSP have serious security implications. Check
https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You
can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
superset_worker_beat | 2023-09-05
09:23:09,595:WARNING:superset.initialization:We haven't found any Content
Security Policy (CSP) defined in the configurations. Please make sure to
configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other
external software. Failing to configure CSP have serious security implications.
Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more
information. You can disable this warning using the
CONTENT_SECURITY_POLICY_WARNING key.
superset_worker_beat | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker_beat | 2023-09-05
09:23:09,598:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker_beat | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker_beat | 2023-09-05
09:23:09,600:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker_beat | 2023-09-05
09:23:09,603:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and
initialization failed: (psycopg2.OperationalError) could not translate host
name "db" to address: Temporary failure in name resolution
superset_worker_beat |
superset_worker_beat | (Background on this error at:
https://sqlalche.me/e/14/e3q8)
superset_worker_beat | Loaded your LOCAL configuration at
[/app/docker/pythonpath_dev/superset_config.py]
superset_worker | logging was configured successfully
superset_worker | 2023-09-05
09:23:09,782:INFO:superset.utils.logging_configurator:logging was configured
successfully
superset_worker | 2023-09-05 09:23:09,784:INFO:root:Configured event
logger of type <class 'superset.utils.log.DBEventLogger'>
superset_worker | We haven't found any Content Security Policy (CSP)
defined in the configurations. Please make sure to configure CSP using the
TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software.
Failing to configure CSP have serious security implications. Check
https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You
can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
superset_worker | 2023-09-05
09:23:09,785:WARNING:superset.initialization:We haven't found any Content
Security Policy (CSP) defined in the configurations. Please make sure to
configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other
external software. Failing to configure CSP have serious security implications.
Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more
information. You can disable this warning using the
CONTENT_SECURITY_POLICY_WARNING key.
superset_worker | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker | 2023-09-05
09:23:09,787:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker | 2023-09-05
09:23:09,790:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_worker | 2023-09-05
09:23:09,793:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and
initialization failed: (psycopg2.OperationalError) could not translate host
name "db" to address: Temporary failure in name resolution
superset_worker |
superset_worker | (Background on this error at:
https://sqlalche.me/e/14/e3q8)
superset_worker | Loaded your LOCAL configuration at
[/app/docker/pythonpath_dev/superset_config.py]
superset_app | logging was configured successfully
superset_app | 2023-09-05
09:23:09,891:INFO:superset.utils.logging_configurator:logging was configured
successfully
superset_app | 2023-09-05 09:23:09,899:INFO:root:Configured event
logger of type <class 'superset.utils.log.DBEventLogger'>
superset_app | We haven't found any Content Security Policy (CSP)
defined in the configurations. Please make sure to configure CSP using the
TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software.
Failing to configure CSP have serious security implications. Check
https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You
can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
superset_app | 2023-09-05
09:23:09,900:WARNING:superset.initialization:We haven't found any Content
Security Policy (CSP) defined in the configurations. Please make sure to
configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other
external software. Failing to configure CSP have serious security implications.
Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more
information. You can disable this warning using the
CONTENT_SECURITY_POLICY_WARNING key.
superset_app | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_app | 2023-09-05
09:23:09,903:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_app | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_app | 2023-09-05
09:23:09,907:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_app | 2023-09-05
09:23:09,920:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and
initialization failed: (psycopg2.OperationalError) could not translate host
name "db" to address: Temporary failure in name resolution
superset_app |
superset_app | (Background on this error at:
https://sqlalche.me/e/14/e3q8)
superset_app | [2023-09-05 09:23:09 +0000] [10] [INFO] Worker
exiting (pid: 10)
superset_app | Loaded your LOCAL configuration at
[/app/docker/pythonpath_dev/superset_config.py]
superset_app | [2023-09-05 09:23:10 +0000] [19] [INFO] Booting
worker with pid: 19
superset_init | logging was configured successfully
superset_init | 2023-09-05
09:23:10,354:INFO:superset.utils.logging_configurator:logging was configured
successfully
superset_init | 2023-09-05 09:23:10,362:INFO:root:Configured event
logger of type <class 'superset.utils.log.DBEventLogger'>
superset_init | We haven't found any Content Security Policy (CSP)
defined in the configurations. Please make sure to configure CSP using the
TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other external software.
Failing to configure CSP have serious security implications. Check
https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more information. You
can disable this warning using the CONTENT_SECURITY_POLICY_WARNING key.
superset_init | 2023-09-05
09:23:10,362:WARNING:superset.initialization:We haven't found any Content
Security Policy (CSP) defined in the configurations. Please make sure to
configure CSP using the TALISMAN_ENABLED and TALISMAN_CONFIG keys or any other
external software. Failing to configure CSP have serious security implications.
Check https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP for more
information. You can disable this warning using the
CONTENT_SECURITY_POLICY_WARNING key.
superset_init | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_init | 2023-09-05
09:23:10,366:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`FILTER_STATE_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_init | Falling back to the built-in cache, that stores
data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_init | 2023-09-05
09:23:10,369:WARNING:superset.utils.cache_manager:Falling back to the built-in
cache, that stores data in the metadata database, for the following cache:
`EXPLORE_FORM_DATA_CACHE_CONFIG`. It is recommended to use `RedisCache`,
`MemcachedCache` or another dedicated caching backend for production deployments
superset_init | 2023-09-05
09:23:10,373:ERROR:flask_appbuilder.security.sqla.manager:DB Creation and
initialization failed: (psycopg2.OperationalError) could not translate host
name "db" to address: Temporary failure in name resolution
superset_init |
Thank you for your help .
Rydka
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
