How come we only serialize data when storing it into the external datastore and not the local cache then?
On Thu, Jun 12, 2014 at 7:56 AM, Romain Leroux <leroux....@gmail.com> wrote: > > https://github.com/nathanmarz/trident-memcached/blob/master/src/jvm/trident/memcached/MemcachedState.java#L62 > localCacheSize sets the size of the LRU cache used locally, so no > CachedMap data are not replicated among all host. > > The idea is that you use this on a grouped stream to store the result of > your aggregation. > So based on the hash (the groupBy fields) all related tuples always go to > the same host. > You want to avoid storing the same cached data in all hosts, otherwise > scaling out is meaningless. > > > > 2014-06-12 6:58 GMT+09:00 Raphael Hsieh <raffihs...@gmail.com>: > > I am imitating the MemCachedState class found here: >> https://github.com/nathanmarz/trident-memcached/blob/master/src/jvm/trident/memcached/MemcachedState.java >> . >> >> I was wondering what the CachedMap is used for. I see that it creates a >> cache layer between the final datastore in order to retrieve values much >> quicker than accessing the datastore every time, however I am unsure about >> a couple details. >> >> Is this cache replicated among all hosts? When I do a 'multiGet' I expect >> to retrieve data I had previously stored. If the cache is specific to each >> host, I wouldn't necessarily get the same data I had most recently stored. >> >> Also, how does this work with Opaque Transactional consistency? It seems >> that in the MemCachedState example we serialize the data once we store it >> to the external datastore, however the data in the cache is not serialized. >> why is this? >> Shouldn't the local cache have the same data as the external datastore ? >> >> thanks >> -- >> Raphael Hsieh >> >> >> >> > > -- Raphael Hsieh
