GitHub user starkmarkus edited a comment on the discussion: storage for tracking rate limits
That warning makes sense, but it is not very user-friendly. Right now the limiter state is stored in process memory. That is fine for development, but it is not a great production default because the counters are lost on restart and are not shared across multiple webserver/API instances. For production, I would expect a shared backend such as Redis or Valkey for the limiter storage. That fits this kind of short-lived, high-churn state much better than the metadata database. If Airflow already exposes a config for the Flask-Limiter storage URI, that seems like the right place to hook this up. Otherwise, this would be a good docs/config improvement. GitHub link: https://github.com/apache/airflow/discussions/68630#discussioncomment-17325734 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
