wulfuric commented on issue #19940:
URL: https://github.com/apache/superset/issues/19940#issuecomment-1196984420
I'm seeing this same behavior as well, the celery worker seems to load the
query fine
```
2022-07-27 16:15:45,322: INFO/MainProcess] Task
load_chart_data_into_cache[4470045e-893b-4591-b9f0-e137680061f1] received
[2022-07-27 16:15:45,323: DEBUG/MainProcess] TaskPool: Apply <function
fast_trace_task at 0x7fd54dd5b790> (args:('load_chart_data_into_cache',
'4470045e-893b-4591-b9f0-e137680061f1', {'lang': 'py', 'task':
'load_chart_data_into_cache', 'id': '4470045e-893b-4591-b9f0-e137680061f1',
'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index':
None, 'retries': 0, 'timelimit': [None, 21600], 'root_id':
'4470045e-893b-4591-b9f0-e137680061f1', 'parent_id': None, 'argsrepr':
"({'channel_id': '672a8009-1db5-45ca-b2eb-6cd2d8f2713b', 'job_id':
'eb001c6c-d2e3-43b4-9b43-af5a84f6fa26', 'user_id': 1, 'status': 'pending',
'errors': [], 'result_url': None}, {'datasource': {'id': 182, 'type': 'table'},
'force': False, 'queries': [{...}], 'form_data': {'datasource': '182__table',
'viz_type': 'big_number_total', 'slice_id': 305, 'url_params': {...},
'granularity_sqla': 'data_dt', 'time_grain_sqla': 'P1D', 'time_range': 'No
filter', 'metric': {...}, 'adhoc_filters': [...], 'header_font
_size': 0.4, 'subheader_font_size': 0.15, 'y_axis_format': 'SMART_NUMBER',
'time_format': 'smart_date', 'extra_form_data': {...}, 'force':... kwargs:{})
The field `timeseries_limit` is deprecated, please use `series_limit`
instead.
[2022-07-27 16:15:45,341: WARNING/MainProcess] The field `timeseries_limit`
is deprecated, please use `series_limit` instead.
[2022-07-27 16:15:45,645: DEBUG/MainProcess] Converted retries value: 3 ->
Retry(total=3, connect=None, read=None, redirect=None, status=None)
[2022-07-27 16:15:45,648: DEBUG/MainProcess] Making request: POST
https://oauth2.googleapis.com/token
[2022-07-27 16:15:45,652: DEBUG/MainProcess] Starting new HTTPS connection
(1): oauth2.googleapis.com:443
[2022-07-27 16:15:45,702: DEBUG/MainProcess]
https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
[2022-07-27 16:15:45,706: DEBUG/MainProcess] Starting new HTTPS connection
(1): bigquery.googleapis.com:443
[2022-07-27 16:15:46,143: DEBUG/MainProcess]
https://bigquery.googleapis.com:443 "POST
/bigquery/v2/projects//jobs?prettyPrint=false HTTP/1.1" 200 None
[2022-07-27 16:15:46,338: DEBUG/MainProcess]
https://bigquery.googleapis.com:443 "GET
/bigquery/v2/projects//queries/c24b2e77-f8e9-441e-9008-366628029d54?maxResults=0&location=US&prettyPrint=false
HTTP/1.1" 200 None
[2022-07-27 16:15:48,358: INFO/MainProcess] Task
load_chart_data_into_cache[4470045e-893b-4591-b9f0-e137680061f1] succeeded in
3.0331106479279697s:
```
But the web server keeps repeatedly hitting the async endpoint, and not
getting any results.
Configs are
```py
class CeleryConfig(object):
broker_url = 'redis://{0}:{1}/0'.format(REDIS_HOST, REDIS_PORT)
imports = (
'superset.sql_lab',
'superset.tasks',
)
result_backend = 'redis://{0}:{1}/0'.format(REDIS_HOST, REDIS_PORT)
worker_log_level = 'DEBUG'
worker_prefetch_multiplier = 1
task_acks_late = False
task_annotations = {
'sql_lab.get_sql_results': {
'rate_limit': '10000/s',
},
'email_reports.send': {
'rate_limit': '1/s',
'time_limit': 120,
'soft_time_limit': 150,
'ignore_result': True,
},
}
CELERY_CONFIG = CeleryConfig
# On Redis
from cachelib.redis import RedisCache
RESULTS_BACKEND = RedisCache(host=REDIS_HOST, port=REDIS_PORT,
key_prefix='superset_results')
# import redis
CACHE_DEFAULT_TIMEOUT = 21600 # Default timeout
GLOBAL_ASYNC_QUERIES_JWT_SECRET = ''
GLOBAL_ASYNC_QUERIES_POLLING_DELAY = 4000
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
"port": REDIS_PORT,
"host": REDIS_HOST,
"password": "",
"db": 0,
"ssl": False,
}
GLOBAL_ASYNC_QUERIES_TRANSPORT = "polling"
CACHE_CONFIG = {
'CACHE_TYPE':'redis', # Use Redis
'CACHE_REDIS_PORT': REDIS_PORT, # Configure port number
'CACHE_REDIS_URL':'redis://{0}:{1}'.format(REDIS_HOST, REDIS_PORT)
}
DATA_CACHE_CONFIG = {
'CACHE_TYPE':'redis', # Use Redis
'CACHE_REDIS_PORT': REDIS_PORT, # Configure port number
'CACHE_REDIS_URL':'redis://{0}:{1}'.format(REDIS_HOST, REDIS_PORT)
}
EXPLORE_FORM_DATA_CACHE_CONFIG = {
'CACHE_TYPE':'redis', # Use Redis
'CACHE_REDIS_PORT': REDIS_PORT, # Configure port number
'CACHE_REDIS_URL':'redis://{0}:{1}'.format(REDIS_HOST, REDIS_PORT)
}
FILTER_STATE_CACHE_CONFIG = {
'CACHE_TYPE':'redis', # Use Redis
'CACHE_REDIS_PORT': REDIS_PORT, # Configure port number
'CACHE_REDIS_URL':'redis://{0}:{1}'.format(REDIS_HOST, REDIS_PORT)
}
FEATURE_FLAGS = {
"ENABLE_TEMPLATE_PROCESSING": True,
"DASHBOARD_NATIVE_FILTERS": True,
"DASHBOARD_CROSS_FILTERS": True,
"VERSIONED_EXPORT": True,
"GLOBAL_ASYNC_QUERIES": True,
}
```
Celery worker running with `"celery --app=superset.tasks.celery_app:app
worker --pool=threads -c 20 --loglevel=DEBUG",`
This is running superset 2.0.0, we may look into using websockets but at
this point hard to invest the time if global async queries might not work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
