GitHub user dosubot[bot] added a comment to the discussion: Caching of Dataset in the redis
<!-- Answer --> To cache datasets in Superset using Redis, you need to ensure that your DATA_CACHE_CONFIG is set up correctly. By default, Superset only caches filter state (using FILTER_STATE_CACHE_CONFIG), not dataset query results. To cache dataset queries, set DATA_CACHE_CONFIG to use Redis (or RedisSentinelCache if you are using Sentinel). Note that Redis Sentinel support is only available in the master branch as of early 2025 and is not included in any official releases yet, so you must be running Superset from the master branch for this to work as expected [[source](https://github.com/apache/superset/issues/33604)]. Your configuration looks mostly correct for Redis Sentinel, but make sure you are running the master branch and that all required parameters (like sentinel master name and authentication, if needed) are set. Also, confirm that DATA_CACHE_CONFIG is being picked up by your deployment and not being overridden elsewhere. Superset constructs cache keys for dataset queries in a way that ensures each unique query (including dataset ID, time range, user context, etc.) gets its own cache entry. Cache timeouts can be set per-query, per-dataset, per-database, or globally via configuration [[source](https://github.com/apache/superset/blob/c09f8f6f7665e503a376926700fa815add6892ca/superset/viz.py)]. If you find that only filter data is being cached and not the dataset results, it is likely because DATA_CACHE_CONFIG is not set up or recognized properly. By default, Superset uses 'NullCache' for dataset caching, which means no caching occurs unless you explicitly configure it [[source](https://github.com/apache/superset/blob/c09f8f6f7665e503a376926700fa815add6892ca/superset/config.py)]. Regarding incremental refresh: Superset does not provide a native mechanism for incremental refresh of datasets, especially for virtual datasets (those defined by a SQL query). All refresh mechanisms in the backend (such as RefreshDatasetCommand or cache warm-up commands) simply re-run the full query. For virtual datasets, any incremental logic must be implemented in your SQL itself, for example by using a WHERE clause on a date column or by leveraging Jinja templating to dynamically filter for new data [[source](https://github.com/apache/superset/blob/c09f8f6f7665e503a376926700fa815add6892ca/docs/docs/configuration/sql-templating.mdx)]. You can use Jinja templating in your virtual dataset SQL to handle time ranges or other dynamic conditions. For example: ```sql SELECT * FROM your_table WHERE dttm_col > '{{ from_dttm }}' AND dttm_col < '{{ to_dttm }}' ``` This approach allows you to control what data is fetched and can be used to simulate incremental refresh by only querying for new or changed data. In summary: ensure DATA_CACHE_CONFIG is set to use Redis and is recognized by your deployment, and implement incremental refresh logic in your virtual dataset SQL using WHERE clauses or Jinja templating. Superset does not provide built-in incremental refresh for virtual datasets [[source](https://github.com/apache/superset/blob/c09f8f6f7665e503a376926700fa815add6892ca/docs/docs/configuration/sql-templating.mdx)]. <!-- Dosu Comment Footer --> *To reply, just mention my name [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/524f8c8e-739f-4c69-a8a2-168fd79f7c33?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/524f8c8e-739f-4c69-a8a2-168fd79f7c33?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/524f8c8e-739f-4c69-a8a2-168fd79f7c33?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/524f8c8e-739f-4c69-a8a2-168fd79f7c33?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/524f8c8e-739f-4c69-a8a2-168fd79f7c33?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/524f8c8e-739f-4c69-a8a2-168fd79f7c33?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/524f8c8e-739f-4c69-a8a2-168fd79f7c33?feedback_type=other)</sup> [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/discussions/33655) GitHub link: https://github.com/apache/superset/discussions/33655#discussioncomment-13336523 ---- This is an automatically sent email for notifications@superset.apache.org. To unsubscribe, please send an email to: notifications-unsubscr...@superset.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org For additional commands, e-mail: notifications-h...@superset.apache.org
