Re: [asterisk-dev] Review Request: Sorcery Caching
+1 On Sat, May 2, 2015 at 7:08 AM, Joshua Colp jc...@digium.com wrote: Greetings all, Based on the feedback I've updated the wiki page[1] with tweaks, additional potential tests, clarification, etc. Cheers, [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev -- [image: Digium logo] Scott Griepentrog Digium, Inc · Software Developer 445 Jan Davis Drive NW · Huntsville, AL 35806 · US direct/fax: +1 256 428 6239 · mobile: +1 256 580 6090 Check us out at: http://digium.com · http://asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
Greetings all, Based on the feedback I've updated the wiki page[1] with tweaks, additional potential tests, clarification, etc. Cheers, [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
On Thu, Apr 30, 2015 at 10:29 AM, Joshua Colp jc...@digium.com wrote: George Joseph wrote: Ok, but will the caching wizard support the C,U,D operations as the memory wizard does?. They could, but the caching infrastructure doesn't currently directly expose the mechanism to do that. I'd err on the side of using observers for it. The core has specific logic for caching because there are no observers on retrieval. As well, the observers are only invoked when the operation actually occurs. Ok, so what would happen if I called ast_sorcery_create with realtime being the concrete store? How does the cache stay synchronized? -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
George Joseph wrote: On Thu, Apr 30, 2015 at 10:29 AM, Joshua Colp jc...@digium.com mailto:jc...@digium.com wrote: George Joseph wrote: Ok, but will the caching wizard support the C,U,D operations as the memory wizard does?. They could, but the caching infrastructure doesn't currently directly expose the mechanism to do that. I'd err on the side of using observers for it. The core has specific logic for caching because there are no observers on retrieval. As well, the observers are only invoked when the operation actually occurs. Ok, so what would happen if I called ast_sorcery_create with realtime being the concrete store? How does the cache stay synchronized? I'll expand the page to detail this. It would add an observer, which gets called when such things successfully occur. It can trigger cache changes as a result of that. Same for update/delete. I think ultimately whether that's useful or not depends on the object type itself. In that scenario if it *has* to be reflected immediately and the object type is seeing heavy write you may end up hurting things versus not caching. If it's marked as stale and eventually reconciled then it's less of an impact. The wiki page was originally written with a focus on read heavy instead. -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
George Joseph wrote: Ok, but will the caching wizard support the C,U,D operations as the memory wizard does?. They could, but the caching infrastructure doesn't currently directly expose the mechanism to do that. I'd err on the side of using observers for it. The core has specific logic for caching because there are no observers on retrieval. As well, the observers are only invoked when the operation actually occurs. Cheers, -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
On Tue, Apr 28, 2015 at 10:28 AM, Joshua Colp jc...@digium.com wrote: Kia ora, I've created a wiki page[1] which details the beginnings of a basic memory based caching wizard for sorcery. Right now while caching is possible using the existing memory wizard it's not possible to define object lifetimes, so once cached it's always pulled from the cache. This wiki page uses the memory wizard as a base but defines options which can tweak the behavior. Going forward this could serve as a basis for other wizards to be created for caching purposes. Some things to consider: 1. How much control and flexibility should we allow? 2. Are there additional mechanisms that should be exposed to allow explicit object expiration? 3. Are the defaults sane? 4. Is there additional testing that should be done? 5. Does anything need additional explanation? Cheers, [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching I didn't see any mention of writes. What happens to create, update and delete operations given that the concrete wizard behind the caching wizard may or may not support them? I also wonder if a variation on this theme might help us consolidate some of the contact and contact_status issues. I'm working on a separate email for that though. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
George Joseph wrote: On Tue, Apr 28, 2015 at 10:28 AM, Joshua Colp jc...@digium.com mailto:jc...@digium.com wrote: Kia ora, I've created a wiki page[1] which details the beginnings of a basic memory based caching wizard for sorcery. Right now while caching is possible using the existing memory wizard it's not possible to define object lifetimes, so once cached it's always pulled from the cache. This wiki page uses the memory wizard as a base but defines options which can tweak the behavior. Going forward this could serve as a basis for other wizards to be created for caching purposes. Some things to consider: 1. How much control and flexibility should we allow? 2. Are there additional mechanisms that should be exposed to allow explicit object expiration? 3. Are the defaults sane? 4. Is there additional testing that should be done? 5. Does anything need additional explanation? Cheers, [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching I didn't see any mention of writes. What happens to create, update and delete operations given that the concrete wizard behind the caching wizard may or may not support them? The behavior remains the same as if there were no caching wizard in place. -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
On Thu, Apr 30, 2015 at 10:13 AM, Joshua Colp jc...@digium.com wrote: George Joseph wrote: On Tue, Apr 28, 2015 at 10:28 AM, Joshua Colp jc...@digium.com mailto:jc...@digium.com wrote: Kia ora, I've created a wiki page[1] which details the beginnings of a basic memory based caching wizard for sorcery. Right now while caching is possible using the existing memory wizard it's not possible to define object lifetimes, so once cached it's always pulled from the cache. This wiki page uses the memory wizard as a base but defines options which can tweak the behavior. Going forward this could serve as a basis for other wizards to be created for caching purposes. Some things to consider: 1. How much control and flexibility should we allow? 2. Are there additional mechanisms that should be exposed to allow explicit object expiration? 3. Are the defaults sane? 4. Is there additional testing that should be done? 5. Does anything need additional explanation? Cheers, [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching I didn't see any mention of writes. What happens to create, update and delete operations given that the concrete wizard behind the caching wizard may or may not support them? The behavior remains the same as if there were no caching wizard in place. Ok, but will the caching wizard support the C,U,D operations as the memory wizard does?. -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
I've had a look at the page now, and here are my thoughts: 1) One thing that isn't really made clear is the interaction between multiple sorcery wizards. In a real-world example, you'd want to hit the memory_cache first and then hit the database afterwards if you couldn't retrieve the object from the cache. Does the order in which items are configured in sorcery.conf dictate the order in which wizards are consulted? Right now, I don't think that's the case. Or is it just that sorcery will automatically see cached stores as being higher priority than others? 2) I agree with Scott's assessment that the global expiration isn't the best idea. The object lifetime option makes much more sense. 3) Object expiration interval is something I feel most people wouldn't really want to set. They'd just hope that something sane was done by default. I think that defaulting to 60 seconds isn't that great a plan either, especially if objects in the cache are set to, say, a 10 second lifetime. Defaulting to some fraction of the object lifetime would work and probably satisfy most people. The value could potentially be updated during a rebalancing operation. Another option is to make use of some sort of timer heap so that in the case of long-lived cached objects, you don't run unneeded checks for object expiration. 4) In addition to the CLI operations, I think equivalent AMI operations would be useful. I also considered the idea of being able to change cache configuration for an object type via AMI/CLI. I'm not sure how useful that would be, and it likely would just create extra contention points where they're really just not needed, so meh. 5) It may be useful to have C-level API calls for invalidating objects/caches. This way, we could implement behavior such as a reload of res_pjsip.so resulting in invalidation of all cached objects owned by res_pjsip.so. 6) The tests are good for testing basic operation of the cache. However, there's not much being done so far with regards to off-nominal code paths (like a user creating a cache with a negative number of maximum objects). First, something needs to be decided regarding how such an error is treated. Do we still create the cache but with a default value configured instead, or do we completely fail to create the cache? Once this is decided, some off-nominal tests with bad configuration should be tested. 7) There doesn't seem to be any way for someone to state that they never want cached objects to automatically become invalidated (i.e. infinite object lifetime). For some installations where configuration is performed through web browsers, for instance, it may be that at the time the config is changed, the system would issue an AMI/CLI command to Asterisk to invalidate the old object. They would essentially always be in charge of telling Asterisk when to invalidate cached objects. I know this is treading close to the old sip prune realtime territory, but I feel like this isn't quite as bad since you would have control over individual objects and object types. 8) My final comment is in regards to the expiration of cached items. Based on wording in tests, it sounds like when the expiration interval arrives, the item is removed from the cache entirely. I wonder if there is a benefit to instead, updating the item when the expiration interval arrives. On the one hand, this has the benefit of meaning that a user of sorcery will always be hitting the cache and never have to actually fall back to a DB lookup. On the other hand, on an idle system, this can result in many pointless DB lookups. So there are tradeoffs, but it's still something to consider. Also, I think Scott pretty much brought up this same idea now that I look back at his response again. On 04/28/2015 11:28 AM, Joshua Colp wrote: Kia ora, I've created a wiki page[1] which details the beginnings of a basic memory based caching wizard for sorcery. Right now while caching is possible using the existing memory wizard it's not possible to define object lifetimes, so once cached it's always pulled from the cache. This wiki page uses the memory wizard as a base but defines options which can tweak the behavior. Going forward this could serve as a basis for other wizards to be created for caching purposes. Some things to consider: 1. How much control and flexibility should we allow? 2. Are there additional mechanisms that should be exposed to allow explicit object expiration? 3. Are the defaults sane? 4. Is there additional testing that should be done? 5. Does anything need additional explanation? Cheers, [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
Re: [asterisk-dev] Review Request: Sorcery Caching
On the subject of maxiumum cache size - I like being able to limit the memory usage, but there is apparently no way to prioritize often used entries. I'm thinking of an example where my cache is full, having gone through every entry, but then a small subset of entries is frequently read which may not have been the first into the cache. If I understand the wording of the wiki article correctly, these would then not benefit from caching, since they would not be allowed in the cache? Also, I'm unclear on the differences between the three object expire/lifetime values. Can you add more descriptive details to each of these values, possibly give an example showing why it is beneficial? My thoughts on cache needs are this: 1) I want to be able to set the maximum memory dedicated to caching, either in MB or in # of entries, but have the caching algorithm take into account the most recently read entries when determining what to hold onto and what to throw away. Something I've just looked up is more likely to used again than something that has been sitting in the cache for a while. To accomplish this each entry needs a timestamp of the last cache hit (when it was last used/read). 2) I want to be able to set two maximum time in cache values: The first is the seconds after which the entry is stale, and the second where it is guarunteed to be removed from the cache and no longer used. Both values are from the time the entry was last obtained from the source (db), requiring a timestamp on entry creation or when it was last refreshed from the source. Before the entry is stale, it can be re-used without any further action needed. When a stale entry is accessed (the first time) it should trigger a background refresh to ensure that changes have not been made, but continue to return a hit from the cache until the second timeout. Background refreshes to be initiated by a separate thread only when there are no primary requests active. The idea here is I can set 300 seconds for stale, and 600 for lifetime -- for a frequently used entry I'm only refreshing from the database every 2.5 minutes and 100% of the requests come immediately from the cache with no database lookup delay to the requesting thread. For any entry that hasn't been read in 5 minutes, it's deleted so the memory utilization goes away also. 3) Flushing all entries on a periodic interval doesn't sound like an option I'd be interested in -- assuming that a maximum time in cache is enforced properly, this seems like an odd duplication. My worry is that dumping all the entries will occasionally occur in the middle of a burst of reads, and thus significantly slow things down. 4) The external notification should be able to forcibly empty the cache, or optionally instead just force all objects to be stale without removing them. This would require adding a stale flag to the entry. 5) The rebalancing option sounds like a good idea, but since it is such an implementation specific value, it would be useful to run some tests and document some statistics on what conditions are needed for it to go from negligable to noticeable improvement, so that people have an idea of when and how to use it. On Tue, Apr 28, 2015 at 11:28 AM, Joshua Colp jc...@digium.com wrote: Kia ora, I've created a wiki page[1] which details the beginnings of a basic memory based caching wizard for sorcery. Right now while caching is possible using the existing memory wizard it's not possible to define object lifetimes, so once cached it's always pulled from the cache. This wiki page uses the memory wizard as a base but defines options which can tweak the behavior. Going forward this could serve as a basis for other wizards to be created for caching purposes. Some things to consider: 1. How much control and flexibility should we allow? 2. Are there additional mechanisms that should be exposed to allow explicit object expiration? 3. Are the defaults sane? 4. Is there additional testing that should be done? 5. Does anything need additional explanation? Cheers, [1] https://wiki.asterisk.org/wiki/display/~jcolp/Sorcery+Caching -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev -- [image: Digium logo] Scott Griepentrog Digium, Inc · Software Developer 445 Jan Davis Drive NW · Huntsville, AL 35806 · US direct/fax: +1 256 428 6239 · mobile: +1 256 580 6090 Check us out at: http://digium.com · http://asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To
Re: [asterisk-dev] Review Request: Sorcery Caching
Scott Griepentrog wrote: On the subject of maxiumum cache size - I like being able to limit the memory usage, but there is apparently no way to prioritize often used entries. I'm thinking of an example where my cache is full, having gone through every entry, but then a small subset of entries is frequently read which may not have been the first into the cache. If I understand the wording of the wiki article correctly, these would then not benefit from caching, since they would not be allowed in the cache? Correct. That approach was based around the concept where you know you'll never have more than 'n' number of objects. Set the limit, allow it to populate, it'll just work. Also, I'm unclear on the differences between the three object expire/lifetime values. Can you add more descriptive details to each of these values, possibly give an example showing why it is beneficial? Will do! My thoughts on cache needs are this: 1) I want to be able to set the maximum memory dedicated to caching, either in MB or in # of entries, but have the caching algorithm take into account the most recently read entries when determining what to hold onto and what to throw away. Something I've just looked up is more likely to used again than something that has been sitting in the cache for a while. To accomplish this each entry needs a timestamp of the last cache hit (when it was last used/read). The con of that is you have to protect the object when on retrieval, introducing a slight contention point for that specific object. 2) I want to be able to set two maximum time in cache values: The first is the seconds after which the entry is stale, and the second where it is guarunteed to be removed from the cache and no longer used. Both values are from the time the entry was last obtained from the source (db), requiring a timestamp on entry creation or when it was last refreshed from the source. Before the entry is stale, it can be re-used without any further action needed. When a stale entry is accessed (the first time) it should trigger a background refresh to ensure that changes have not been made, but continue to return a hit from the cache until the second timeout. Background refreshes to be initiated by a separate thread only when there are no primary requests active. The idea here is I can set 300 seconds for stale, and 600 for lifetime -- for a frequently used entry I'm only refreshing from the database every 2.5 minutes and 100% of the requests come immediately from the cache with no database lookup delay to the requesting thread. For any entry that hasn't been read in 5 minutes, it's deleted so the memory utilization goes away also. Hrm, without sorcery architectural changes it would be rather difficult to implement. I'll look at it. 3) Flushing all entries on a periodic interval doesn't sound like an option I'd be interested in -- assuming that a maximum time in cache is enforced properly, this seems like an odd duplication. My worry is that dumping all the entries will occasionally occur in the middle of a burst of reads, and thus significantly slow things down. Understood. 4) The external notification should be able to forcibly empty the cache, or optionally instead just force all objects to be stale without removing them. This would require adding a stale flag to the entry. 5) The rebalancing option sounds like a good idea, but since it is such an implementation specific value, it would be useful to run some tests and document some statistics on what conditions are needed for it to go from negligable to noticeable improvement, so that people have an idea of when and how to use it. I have some experience with that actually when I added support for it to the res_sorcery_config module. You can reduce lookup times quite a lot depending on the number of objects. Could definitely be experimented with, though. -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com www.asterisk.org -- _ -- Bandwidth and Colocation Provided by http://www.api-digital.com -- asterisk-dev mailing list To UNSUBSCRIBE or update options visit: http://lists.digium.com/mailman/listinfo/asterisk-dev
