Re: [Django] #33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult
#33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult -+- Reporter: bpicolo |Owner: nobody Type: | Status: new Cleanup/optimization | Component: Core (Cache system) | Version: 4.0 Severity: Normal | Resolution: Keywords: | Triage Stage: | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Description changed by bpicolo: Old description: > Hello! First Django interaction, so please let me know how better I can > give info here. > > I discovered unintentionally that there's no default socket_timeout set > for Redis cache connections. This is an issue on it's own (unsafe default > that's a particularly hard issue to track down), but in trying to set > those timeouts, I hit some tough edges of documentation/usage. > > The cache documentation says that the OPTIONS object is passed to the > third-party connection class for connections backed by third-party > libraries > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20 > for connections backed by third-party libraries]. This is reaffirmed for > the redis cache specifically > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors > here]. > > Right now, though, these OPTIONS are passed to the > [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 > first-party RedisCacheClient object], so you can't pass in options > expected by the redis connection pool [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1283 link]. In that sense, > RedisCacheClient is being treated as a "first-party" cache according to > these docs, but it defers directly to the redis library. > > One could do a variety of subclassing to make this work, but there's > quite a few layers of indirection here, so it's hard to identify exactly > what one should subclass in order to get arguments to the connection > class appropriately. > > (For now, my workaround is take advantage of the from_url behavior where > the connection class [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1274 pulls arguments out of the > connection query string], but this isn't straightforward and feels > brittle to django implementation details). > > There are a couple of options here – revising documentation or changing > behavior (or both). > > In my ideal case, it would be great for it to be easy to pass down > explicit socket and other timeouts. To me, that suggests updating > behavior is ideal, because the documentation would potentially lead folk > down a path of tough to navigate behavior. New description: Hello! First Django interaction, so please let me know how better I can give info here. I discovered unintentionally that there's no default socket_timeout set for Redis cache connections. This is an issue on it's own (unsafe default that's a particularly hard issue to track down), but in trying to set those timeouts, I hit some tough edges of documentation/usage. The cache documentation says that the OPTIONS object is passed to the third-party connection class for connections backed by third-party libraries [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20 for connections backed by third-party libraries]. This is reaffirmed for the redis cache specifically [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors here]. Right now, though, these OPTIONS are passed to the [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 first-party RedisCacheClient object], so you can't pass in options expected by the redis connection pool [https://github.com/redis/redis- py/blob/master/redis/connection.py#L1283 link]. In that sense, RedisCacheClient is being treated as a "first-party" cache according to these docs, but it defers directly to the redis libr
Re: [Django] #33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult
#33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult -+- Reporter: bpicolo |Owner: nobody Type: | Status: new Cleanup/optimization | Component: Core (Cache system) | Version: 4.0 Severity: Normal | Resolution: Keywords: | Triage Stage: | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Description changed by bpicolo: Old description: > Hello! First Django interaction, so please let me know how better I can > give info here. > > I discovered unintentionally that there's no default socket_timeout set > for Redis cache connections. This is an issue on it's own (unsafe default > that's a particularly hard issue to track down), but in trying to set > those timeouts, I hit some tough edges of documentation/usage. > > The cache documentation says that the OPTIONS object is passed to the > third-party connection class for connections backed by third-party > libraries > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20 > for connections backed by third-party libraries]. This is reaffirmed for > the redis cache specifically > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors > here]. > > Right now, though, these OPTIONS are passed to the > [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 > first-party RedisCacheClient object], so you can't pass in options > expected by the redis connection pool [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1283 link]. In that sense, > RedisCacheClient is being treated as a "first-party" cache according to > these docs, but it defers directly to the redis library. > > One could do a variety of subclassing to make this work, but there's > quite a few layers of indirection here, so it's hard to identify exactly > what one should subclass in order to get arguments to the connection > class appropriately. > > (For now, my workaround is take advantage of the from_url behavior where > the connection class [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1274 pulls arguments out of the > connection query string], but this isn't straightforward and feels > brittle). > > There are a couple of options here – revising documentation or changing > behavior (or both). > > In my ideal case, it would be great for it to be easy to pass down > explicit socket and other timeouts. To me, that suggests updating > behavior is ideal, because the documentation would potentially lead folk > down a path of tough to navigate behavior. New description: Hello! First Django interaction, so please let me know how better I can give info here. I discovered unintentionally that there's no default socket_timeout set for Redis cache connections. This is an issue on it's own (unsafe default that's a particularly hard issue to track down), but in trying to set those timeouts, I hit some tough edges of documentation/usage. The cache documentation says that the OPTIONS object is passed to the third-party connection class for connections backed by third-party libraries [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20 for connections backed by third-party libraries]. This is reaffirmed for the redis cache specifically [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors here]. Right now, though, these OPTIONS are passed to the [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 first-party RedisCacheClient object], so you can't pass in options expected by the redis connection pool [https://github.com/redis/redis- py/blob/master/redis/connection.py#L1283 link]. In that sense, RedisCacheClient is being treated as a "first-party" cache according to these docs, but it defers directly to the redis library. One could do a variety of
Re: [Django] #33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult
#33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult -+- Reporter: bpicolo |Owner: nobody Type: | Status: new Cleanup/optimization | Component: Core (Cache system) | Version: 4.0 Severity: Normal | Resolution: Keywords: | Triage Stage: | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Description changed by bpicolo: Old description: > Hello! First Django interaction, so please let me know how better I can > give info here. > > I discovered unintentionally that there's no default socket_timeout set > for Redis cache connections. This is an issue on it's own (unsafe default > that's a particularly hard issue to track down), but in trying to set > those timeouts, I hit some tough edges of documentation/usage. > > The cache documentation says that the OPTIONS object is passed to the > third-party connection class for connections backed by third-party > libraries > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library > is passed to the underlying library connection object]. This is > reaffirmed for the redis cache specifically > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors > here]. > > Right now, though, these OPTIONS are passed to the > [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 > first-party RedisCacheClient object], so you can't pass in options > expected by the redis connection pool [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1283 link]. In that sense, > RedisCacheClient is being treated as a "first-party" cache according to > these docs, but it defers directly to the redis library. > > One could do a variety of subclassing to make this work, but there's > quite a few layers of indirection here, so it's hard to identify exactly > what one should subclass in order to get arguments to the connection > class appropriately. > > (For now, my workaround is take advantage of the from_url behavior where > the connection class [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1274 pulls arguments out of the > connection query string], but this isn't straightforward and feels > brittle). > > There are a couple of options here – revising documentation or changing > behavior (or both). > > In my ideal case, it would be great for it to be easy to pass down > explicit socket and other timeouts. To me, that suggests updating > behavior is ideal, because the documentation would potentially lead folk > down a path of tough to navigate behavior. New description: Hello! First Django interaction, so please let me know how better I can give info here. I discovered unintentionally that there's no default socket_timeout set for Redis cache connections. This is an issue on it's own (unsafe default that's a particularly hard issue to track down), but in trying to set those timeouts, I hit some tough edges of documentation/usage. The cache documentation says that the OPTIONS object is passed to the third-party connection class for connections backed by third-party libraries [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20 for connections backed by third-party libraries]. This is reaffirmed for the redis cache specifically [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors here]. Right now, though, these OPTIONS are passed to the [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 first-party RedisCacheClient object], so you can't pass in options expected by the redis connection pool [https://github.com/redis/redis- py/blob/master/redis/connection.py#L1283 link]. In that sense, RedisCacheClient is being treated as a "first-party" cache according to these docs, but it defers directly to the redis library. One could do
Re: [Django] #33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult
#33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult -+- Reporter: bpicolo |Owner: nobody Type: | Status: new Cleanup/optimization | Component: Core (Cache system) | Version: 4.0 Severity: Normal | Resolution: Keywords: | Triage Stage: | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Description changed by bpicolo: Old description: > Hello! First Django interaction, so please let me know how better I can > give info here. > > I discovered unintentionally that there's no default socket_timeout set > for Redis cache connections. This is an issue on it's own (unsafe default > that's a particularly hard issue to track down), but in trying to set > those timeouts, I hit some tough edges of documentation/usage. > > The cache documentation says that, for caches backed by third-party > libraries, the OPTIONS object > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library > is passed to the underlying library connection object]. This is > reaffirmed for the redis cache specifically > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors > here]. > > Right now, though, these OPTIONS are passed to the > [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 > first-party RedisCacheClient object], so you can't pass in options > expected by the redis connection pool [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1283 link]. > > One could do a variety of subclassing to make this work, but there's > quite a few layers of indirection here, so it's hard to identify exactly > what one should subclass in order to get arguments to the connection > class appropriately. > > (For now, my workaround is take advantage of the from_url behavior where > the connection class [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1274 pulls arguments out of the > connection query string], but this isn't straightforward and feels > brittle). > > There are a couple of options here – revising documentation or changing > behavior (or both). > > In my ideal case, it would be great for it to be easy to pass down > explicit socket and other timeouts. To me, that suggests updating > behavior is ideal, because the documentation would potentially lead folk > down a path of tough to navigate behavior. New description: Hello! First Django interaction, so please let me know how better I can give info here. I discovered unintentionally that there's no default socket_timeout set for Redis cache connections. This is an issue on it's own (unsafe default that's a particularly hard issue to track down), but in trying to set those timeouts, I hit some tough edges of documentation/usage. The cache documentation says that the OPTIONS object is passed to the third-party connection class for connections backed by third-party libraries [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library is passed to the underlying library connection object]. This is reaffirmed for the redis cache specifically [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors here]. Right now, though, these OPTIONS are passed to the [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 first-party RedisCacheClient object], so you can't pass in options expected by the redis connection pool [https://github.com/redis/redis- py/blob/master/redis/connection.py#L1283 link]. In that sense, RedisCacheClient is being treated as a "first-party" cache according to these docs, but it defers directly to the redis library. One could do a variety of subclassing to make this work, but there's quite a few layers of indirection here, so it's hard to identify exactly what one should subclass in order to get arguments to t
Re: [Django] #33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult
#33681: Redis client OPTIONS don't work as documented, which makes setting Redis timeouts difficult -+- Reporter: bpicolo |Owner: nobody Type: | Status: new Cleanup/optimization | Component: Core (Cache system) | Version: 4.0 Severity: Normal | Resolution: Keywords: | Triage Stage: | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Description changed by bpicolo: Old description: > Hello! First Django interaction, so please let me know how better I can > give info here. > > I discovered unintentionally that there's no default socket_timeout set > for Redis cache connections. This is an issue on it's own (unsafe default > that's a particularly hard issue to track down), but in trying to set > those timeouts, I hit some tough edges of documentation/usage. > > The cache documentation says that, for caches backed by third-party > libraries, the OPTIONS object > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library > is passed to the underlying library connection object]. This is > reaffirmed for the redis cache specifically > [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors > here]. > > Right now, though, these OPTIONS are passed to the > [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 > first-party RedisCacheClient object], so you can't pass in options > expected by the redis connection pool [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1283 link]. > > One could do a variety of subclassing to make this work, but there's > quite a few layers of indirection here, so it's hard to identify exactly > what one should subclass in order to get objects to the connection class > appropriately. > > (For now, my workaround is take advantage of the from_url behavior where > the connection class [https://github.com/redis/redis- > py/blob/master/redis/connection.py#L1274 pulls arguments out of the > connection query string], but this isn't straightforward and feels > brittle). > > There are a couple of options here – revising documentation or changing > behavior (or both). > > In my ideal case, it would be great for it to be easy to pass down > explicit socket and other timeouts. To me, that suggests updating > behavior is ideal, because the documentation would potentially lead folk > down a path of tough to navigate behavior. New description: Hello! First Django interaction, so please let me know how better I can give info here. I discovered unintentionally that there's no default socket_timeout set for Redis cache connections. This is an issue on it's own (unsafe default that's a particularly hard issue to track down), but in trying to set those timeouts, I hit some tough edges of documentation/usage. The cache documentation says that, for caches backed by third-party libraries, the OPTIONS object [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=cache%20backends%20backed%20by%20a%20third%2Dparty%20library%20will%20pass%20their%20options%20directly%20to%20the%20underlying%20cache%20library is passed to the underlying library connection object]. This is reaffirmed for the redis cache specifically [https://docs.djangoproject.com/en/4.0/topics/cache/#:~:text=The%20Memcached%20and%20Redis%20backends%20pass%20the%20contents%20of%20OPTIONS%20as%20keyword%20arguments%20to%20the%20client%20constructors here]. Right now, though, these OPTIONS are passed to the [https://github.com/django/django/blob/7119f40c9881666b6f9b5cf7df09ee1d21cc8344/django/core/cache/backends/redis.py#L30 first-party RedisCacheClient object], so you can't pass in options expected by the redis connection pool [https://github.com/redis/redis- py/blob/master/redis/connection.py#L1283 link]. One could do a variety of subclassing to make this work, but there's quite a few layers of indirection here, so it's hard to identify exactly what one should subclass in order to get arguments to the connection class appropriately. (For now, my workaround is take advantage of the from_url behavior where the connection class [https://github.com/redis/redis- py/blob/master/redis/connection