alexhockley opened a new issue #3636: Unable to configure Async worker with RabbitMQ URL: https://github.com/apache/incubator-superset/issues/3636 ### Server information Ubuntu 16.04.3 LTS on Amazon EC2 ### Superset version 0.20.3 ### Expected results Async query finishes ### Actual results Async query hangs. gunicorn stdout output on query run: `2017-10-10 15:58:24,179:INFO:root:[stats_logger] (incr) sql_json 2017-10-10 15:58:24,183:INFO:root:Parsing with sqlparse statement SELECT * FROM main.ab_permission 2017-10-10 15:58:24,215:INFO:root:Triggering query_id: 13 2017-10-10 15:58:24,215:INFO:root:Running query on a Celery worker /home/ubuntu/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:596: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage. 'storage.' % (dialect.name, dialect.driver)) 2017-10-10 15:58:24,240:DEBUG:amqp:Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2015 Pivotal Software, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'per_consumer_qos': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'cluster_name': u'[email protected]', u'platform': u'Erlang/OTP', u'version': u'3.5.7'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US'] 2017-10-10 15:58:24,240:DEBUG:amqp:Open OK! 2017-10-10 15:58:24,241:DEBUG:amqp:using channel_id: 1 2017-10-10 15:58:24,241:DEBUG:amqp:Channel open 99.225.182.133 - - [10/Oct/2017:15:58:24 +0000] "POST /superset/sql_json/ HTTP/1.1" 202 546 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" 2017-10-10 15:58:25,602:INFO:root:[stats_logger] (incr) queries 99.225.182.133 - - [10/Oct/2017:15:58:25 +0000] "GET /superset/queries/1507650390378.714 HTTP/1.1" 200 2201 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" 2017-10-10 15:58:27,595:INFO:root:[stats_logger] (incr) queries 99.225.182.133 - - [10/Oct/2017:15:58:27 +0000] "GET /superset/queries/1507651099210.269 HTTP/1.1" 200 550 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" 2017-10-10 15:58:29,592:INFO:root:[stats_logger] (incr) queries 99.225.182.133 - - [10/Oct/2017:15:58:29 +0000] "GET /superset/queries/1507651099210.269 HTTP/1.1" 200 550 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"` tailing /var/log/rabbitmq/[email protected] has: `=INFO REPORT==== 10-Oct-2017::15:58:24 === accepting AMQP connection <0.525.0> (127.0.0.1:37060 -> 127.0.0.1:5672)` /tml/sqllab_cache exists and should have write permissions ### Steps to reproduce - Boot fresh server on Amazon EC2 - Follow install instructions located on https://superset.incubator.apache.org/installation.html - Configure gunicorn (doesnt work with regular runserver either though) - Install RabbitMQ - Run `gunicorn --error-logfile - --access-logfile - -w 8 -k gevent -b 0.0.0.0:8080 --timeout 120 --limit-request-line 0 --limit-request-field_size 0 --forwarded-allow-ips="*" superset:app` to start the web server - Connect to <public ip>:8080, login, configure the sample database to use async workers instead of sync (it works fine with sync workers) - Go to SQL Lab and run a simple SELECT * FROM main.ab_permission Using the following superset custom config file, located in the same directory i run the gunicorn command from `#--------------------------------------------------------- # Superset specific config #--------------------------------------------------------- ROW_LIMIT = 5000 SUPERSET_WORKERS = 4 SUPERSET_WEBSERVER_PORT = 8088 #--------------------------------------------------------- #--------------------------------------------------------- # Flask App Builder configuration #--------------------------------------------------------- # Your App secret key SECRET_KEY = '\2\1thisismyscretkey\1\2\e\y\y\h' # The SQLAlchemy connection string to your database backend # This connection defines the path to the database that stores your # superset metadata (slices, connections, tables, dashboards, ...). # Note that the connection information to connect to the datasources # you want to explore are managed directly in the web UI #SQLALCHEMY_DATABASE_URI = 'sqlite:////path/to/superset.db' # Flask-WTF flag for CSRF WTF_CSRF_ENABLED = True # Add endpoints that need to be exempt from CSRF protection WTF_CSRF_EXEMPT_LIST = [] # Set this API key to enable Mapbox visualizations MAPBOX_API_KEY = '' from werkzeug.contrib.cache import FileSystemCache RESULTS_BACKEND = FileSystemCache('/tmp/sqllab_cache', default_timeout=60*24*7) ` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
With regards, Apache Git Services
