GitHub user dosubot[bot] added a comment to the discussion: Superset pod crashes and high download latency with large Presto query results (>5M rows) — is streaming download supported?
<!-- Answer --> I found a similar unsolved discussion titled [Does Superset support streaming results?](https://github.com/apache/superset/discussions/33030), which might be relevant to your question about streaming downloads <sup>[[1]](https://github.com/apache/superset/discussions/33030)</sup>. Regarding your questions: 1. **Long Fetch and Download Times**: Superset's performance is heavily dependent on the underlying database's ability to execute queries and return results quickly. If the database is slow, Superset's performance will also be slow. Additionally, the caching configuration might not be optimized for your use case <sup>[[2]](https://github.com/apache/superset/blob/master/docs/docs/faq.mdx)</sup>. 2. **Streaming Download Support**: There is no clear indication that Superset supports streaming downloads like Hue. This feature might not be available in Superset <sup>[[1]](https://github.com/apache/superset/discussions/33030)</sup>. 3. **Downloading Large Datasets**: Superset has a maximum data size limit for downloads, defined by the `SQLLAB_PAYLOAD_MAX_MB` configuration. If the serialized payload size exceeds this limit, an error is raised. The default `ROW_LIMIT` is set to 50,000, which might affect large downloads <sup>[[3]](https://github.com/apache/superset/blob/master/superset/sql_lab.py)</sup><sup>[[4]](https://github.com/apache/superset/blob/master/superset/config.py)</sup>. 4. **Optimizing Performance and Preventing Crashes**: While specific best practices for preventing pod crashes aren't detailed, optimizing caching configurations and ensuring that your database and Superset are properly scaled for large datasets might help. Consider using a more robust caching backend like Redis, as you are already doing <sup>[[5]](https://github.com/apache/superset/blob/master/tests/integration_tests/cache_tests.py)</sup>. 5. **Bypassing UI for Downloads**: This approach might help if the UI is a bottleneck, but it would require custom development to implement. For best practices and optimizations, consider reviewing your caching strategy and ensuring that your infrastructure is scaled appropriately for the data sizes you are handling. <!-- Dosu Comment Footer --> *To continue talking to [Dosu](https://dosu.dev), mention @dosu.* --- <sup>Help Dosu learn! Give it feedback: [Great Response](https://app.dosu.dev/response-feedback/6fcaba33-02df-4916-95f4-8dee414d7fc1?feedback_type=great_response) | [Irrelevant Answer](https://app.dosu.dev/response-feedback/6fcaba33-02df-4916-95f4-8dee414d7fc1?feedback_type=irrelevant_answer) | [Incorrect Sources](https://app.dosu.dev/response-feedback/6fcaba33-02df-4916-95f4-8dee414d7fc1?feedback_type=incorrect_sources) | [Too Verbose](https://app.dosu.dev/response-feedback/6fcaba33-02df-4916-95f4-8dee414d7fc1?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/6fcaba33-02df-4916-95f4-8dee414d7fc1?feedback_type=hallucination) | [Bug Report](https://app.dosu.dev/response-feedback/6fcaba33-02df-4916-95f4-8dee414d7fc1?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/6fcaba33-02df-4916-95f4-8dee414d7fc1?feedback_type=other)</sup> GitHub link: https://github.com/apache/superset/discussions/33243#discussioncomment-12947088 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
