nogreintel opened a new issue, #20367: URL: https://github.com/apache/superset/issues/20367
I have custom code that automates the creation of superset dashboards. It does this using the [superset-client-library](https://github.com/opus-42/superset-api-client). A dashboard is created, and as a result, around 20+ charts are created and added to the dashboard. When running with the following settings, I have 2-3 failed dashboards for 10 concurrent requests: ``` replicaCount: 4 extraEnv: # Extend timeout to allow long running queries. GUNICORN_TIMEOUT: 300 SERVER_WORKER_AMOUNT: 2 SERVER_THREADS_AMOUNT: 20 resources: limits: cpu: 4000m memory: 2048Mi requests: cpu: 2000m memory: 1024Mi ``` As opposed to an earlier configuration that had 6-8 failed dashboards: ``` replicaCount: 2 extraEnv: # Extend timeout to allow long running queries. GUNICORN_TIMEOUT: 300 SERVER_WORKER_AMOUNT: 2 SERVER_THREADS_AMOUNT: 20 resources: limits: cpu: 1000m memory: 2048Mi requests: cpu: 500m memory: 1024Mi ``` Further attempts to increase all of the above values do not appear to improve the results beyond the benchmark set by the most optimal configuration at the top. I did also try increasing GUNICORN_KEEPALIVE, and this did seem to get me up to 10 successful requests, but then 502 started causing 502 errors all the time. #### How to reproduce the bug One can use the code in [superset-client-library](https://github.com/opus-42/superset-api-client) or curl the superset API, and stress test it. ### Expected results Successful automated dashboard creation and ways to properly scale superset to handle concurrent creation of dashboards. ### Actual results When too many concurrent requests occur, I receive the following error: ``` [ERROR] HTTPError: 502 Server Error: Bad Gateway for url: https://supersetwebapp.com/api/v1/dashboard/import/ Traceback (most recent call last): File "/app/superset_etl/create_dashboard.py", line 227, in main dashboard_url = dashboard_import(dashboard_json_path, chart_pickle_path, experiment_id, iteration, svr_info) File "/app/superset_etl/create_dashboard.py", line 180, in dashboard_import import_success = client.dashboards.import_file(dashboard_json_path) File "/var/lang/lib/python3.8/site-packages/supersetapiclient/base.py", line 381, in import_file response.raise_for_status() File "/var/lang/lib/python3.8/site-packages/requests/models.py", line 960, in raise_for_status raise HTTPError(http_error_msg, response=self) ``` #### Screenshots If applicable, add screenshots to help explain your problem. ### Environment (please complete the following information): - superset version: `1.5.1` - python version: `3.7` ### Checklist Make sure to follow these steps before submitting your issue - thank you! - [x] I have checked the superset logs for python stacktraces and included it here as text if there are any. - [ ] I have reproduced the issue with at least the latest released version of superset. - [x] I have checked the issue tracker for the same issue and I haven't found one similar. ### Questions What are the ideal configurations in order to help scale up the superset ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
