Re: Should I store offline calculation results in the cache?

2017-05-29 Thread Antonis Christofides
Hello, thanks to everyone who replied. Here are some conclusions of mine: Today's filebased-cache code seems to be suffering from the same problems it was suffering 7 years ago. Every time you .set() the cache it asks the OS to provide a list of files, just for counting them (for the purpose of

Re: Should I store offline calculation results in the cache?

2017-05-28 Thread Melvyn Sopacua
On Saturday 27 May 2017 12:25:17 Antonis Christofides wrote: > The question is where to store the results of the calculation. I > thought that since they are re-creatable, the cache would be the > appropriate place. However, there is a difference with the more > common use of the cache: they are

Re: Should I store offline calculation results in the cache?

2017-05-27 Thread Alex Heyden
Seven years ago, that may very well have been a true statement. I wouldn't stand by it today. Disk caching should be perfectly stable, even if it's a pretty poor solution for the average use case. The part that hasn't changed is memcached. Memcached should be everyone's default caching solution,

Re: Should I store offline calculation results in the cache?

2017-05-27 Thread Alexandre Pereira da Silva
Redis caching is a good solution for this. You can have persistence, it's fast and only limited by your available RAM. Your implementation should be more robust about missing itens from the cache. If for any reason the cache is lost, your code should rebuild the values in a background task. On

Should I store offline calculation results in the cache?

2017-05-27 Thread Antonis Christofides
Hello all, I have an application that calculates and tells you whether a specific crop at a specific piece of land needs to be irrigated, and how much. The calculation lasts for a few seconds, so I'm doing it offline with Celery. Every two hours new meteorological data comes in and all the pieces