dosubot[bot] commented on issue #38012: URL: https://github.com/apache/superset/issues/38012#issuecomment-3908243785
<!-- Answer --> You're right, my apologies for the confusion. The Dataset Editor modal has a different issue than what those SQL Lab PRs addressed. The behavior you're seeing (error shown but dataset actually saves) is caused by a **two-stage save process** in the [Dataset PUT endpoint](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/datasets/api.py#L364-L447): 1. **Stage 1**: `UpdateDatasetCommand` saves the dataset successfully ✅ 2. **Stage 2**: `RefreshDatasetCommand` runs immediately after to [sync column metadata](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/commands/dataset/refresh.py#L43-L48) by executing the SQL with `LIMIT 1` The error occurs in Stage 2 because [`get_virtual_table_metadata()`](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/connectors/sqla/utils.py#L97-L130) processes Jinja templates **without runtime context** — functions like `filter_values()` return empty lists, generating invalid SQL like `WHERE col IN ()`. This is a known issue tracked in [#33591](https://github.com/apache/superset/issues/33591) and [#25839](https://github.com/apache/superset/issues/25839). The core problem is that Dataset Editor validates at save time without runtime context, whereas SQL Lab validates at execution time with full context. This appears to be a separate bug from the SQL Lab fixes and may warrant its own fix specifically for the Dataset Editor's metadata refresh flow. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/0a831891-b727-4929-833f-6b558ac0558b?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/0a831891-b727-4929-833f-6b558ac0558b?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/0a831891-b727-4929-833f-6b558ac0558b?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/0a831891-b727-4929-833f-6b558ac0558b?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/0a831891-b727-4929-833f-6b558ac0558b?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/0a831891-b727-4929-833f-6b558ac0558b?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/0a831891-b727-4929-833f-6b558ac0558b?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=You%27re%20right%2C%20my%20apologies%20for%20the%20confusion.%20The%20Dataset%20Editor%20modal%20has%20a%20different%20issue%20than%20what%20those%20SQL%20Lab%20PRs%20addressed.%0A%0AThe%20behavior%20you%27re%20seeing%20%28error%20shown%20but%20dataset%20actually%20saves%29%20is%20caused%20by%20a%20%2A%2Atwo-stage%20save%20process%2A%2A%20in%20the%20%5BDataset%20PUT%20endpoint%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/datasets/api.py%23L364-L447%29%3A%0A%0A1.%20%2A%2AStage%201%2A%2A%3A%20%60UpdateDatasetCommand%60%20saves%20the%20dataset%20successfully%20%E2%9C%85%0A2.%20%2A%2AStage%202%2A%2A%3A%20%60RefreshDatasetCommand%60%20runs%20immediately%20after%20to%20%5Bsync%20column%20metadata%5D%28h ttps%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/commands/dataset/refresh.py%23L43-L48%29%20by%20executing%20the%20SQL%20with%20%60LIMIT%201%60%0A%0AThe%20error%20occurs%20in%20Stage%202%20because%20%5B%60get_virtual_table_metadata%28%29%60%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/connectors/sqla/utils.py%23L97-L130%29%20processes%20Jinja%20templates%20%2A%2Awithout%20runtime%20context%2A%2A%20%E2%80%94%20functions%20like%20%60filter_values%28%29%60%20return%20empty%20lists%2C%20generating%20invalid%20SQL%20like%20%60WHERE%20col%20IN%20%28%29%60.%0A%0AThis%20is%20a%20known%20issue%20tracked%20in%20%5B%2333591%5D%28https%3A//github.com/apache/superset/issues/33591%29%20and%20%5B%2325839%5D%28https%3A//github.com/apache/superset/issues/25839%29.%20The%20core%20problem%20is%20that%20Dataset%20Editor%20validates%20at%20save%20time%20without%20runtime%20context%2C%20whereas%20SQL%20Lab%20val idates%20at%20execution%20time%20with%20full%20context.%0A%0AThis%20appears%20to%20be%20a%20separate%20bug%20from%20the%20SQL%20Lab%20fixes%20and%20may%20warrant%20its%20own%20fix%20specifically%20for%20the%20Dataset%20Editor%27s%20metadata%20refresh%20flow.) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/38012) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
