mikebridge commented on PR #42760:
URL: https://github.com/apache/superset/pull/42760#issuecomment-5184966370
### Manual local testing
The clearest manual test uses two different semantic queries where the first
result safely contains the second. This distinguishes containment reuse from
Superset's ordinary exact-query cache.
No built-in production semantic provider opts into this cache yet, so this
test
requires a local provider or extension implementing the new cache contract.
#### 1. Start Redis
```bash
docker run --rm --name superset-semantic-redis \
-p 6379:6379 redis:7-alpine
```
#### 2. Configure Superset
Add the following to `superset_config.py`:
```python
FEATURE_FLAGS = {
"SEMANTIC_LAYERS": True,
"SEMANTIC_LAYER_CONTAINMENT_CACHE": True,
}
DATA_CACHE_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_REDIS_HOST": "127.0.0.1",
"CACHE_REDIS_PORT": 6379,
"CACHE_REDIS_DB": 1,
"CACHE_DEFAULT_TIMEOUT": 300,
}
DISTRIBUTED_COORDINATION_CONFIG = {
"CACHE_TYPE": "RedisCache",
"CACHE_REDIS_HOST": "127.0.0.1",
"CACHE_REDIS_PORT": 6379,
"CACHE_REDIS_DB": 2,
"CACHE_REDIS_PASSWORD": "",
}
```
When Superset runs in Docker, use the Redis service/container hostname
instead
of `127.0.0.1`. Restart every web and worker process after changing
configuration.
#### 3. Opt in the semantic provider
The provider needs the equivalent of:
```python
from superset_core.semantic_layers.layer import (
SemanticCacheCapabilities,
SemanticCacheIdentityMaterial,
SemanticCacheResponsibility,
SemanticCacheScope,
)
semantic_cache_responsibility = SemanticCacheResponsibility.SUPERSET
semantic_cache_scope = SemanticCacheScope.EXECUTION_CONTEXT
semantic_cache_capabilities = SemanticCacheCapabilities(
comparisons=True,
membership=True,
nulls=True,
pattern_escape="\\",
)
def get_semantic_cache_provider_identity(self):
return SemanticCacheIdentityMaterial(
{"provider_version": "local-test-v1", "catalog": self.config.catalog}
)
def get_semantic_cache_context_identity(self, context):
return SemanticCacheIdentityMaterial({"tenant": "local-test"})
```
#### 4. Exercise containment
1. Create a chart from an opted-in semantic view, grouped by `country` and
`city`, using an additive metric such as `SUM(revenue)`.
2. Run it and confirm that the semantic provider receives the request.
3. Remove `city`, leaving the chart grouped only by `country`.
4. Run it again.
The second query differs from the first, but the first result contains enough
detail to answer it. Superset should roll up the cached rows without another
provider request.
#### 5. Verify the result
Inspect the chart-data response in browser developer tools. It should
contain:
```json
{
"semantic_cache_hit": true
}
```
The cache pill tooltip should say **Loaded from semantic cache**. Provider
logs
or a temporary request counter should show that only the first query reached
the
provider. Confirm that the country totals exactly match a forced provider
query.
#### 6. Negative controls
- Click the cache pill to force-refresh: the provider should run and
`semantic_cache_hit` should be false.
- Request a metric absent from the cached result: the provider should run.
- Disable `SEMANTIC_LAYER_CONTAINMENT_CACHE` and restart: both requests
should
reach the provider.
- Remove `DISTRIBUTED_COORDINATION_CONFIG` and restart: containment should be
disabled without breaking semantic queries.
- With `EXECUTION_CONTEXT`, repeat as another user: that user's first request
should miss rather than reuse the other user's result.
The same behavior has a quick non-UI smoke test:
```bash
source ~/venv/superset/bin/activate
pytest -q tests/unit_tests/semantic_layers/cache_integration_test.py
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
