I've been reading over the storage code and have come to the conclusion
that the async aspects of the storage framework should be removed.
Whenever one introduces an async pattern you have to give a lot of
consideration to its use, benefits, and impact. Within the context of
ACS and given the current state of its code, I do not think it will be
possible to realize any benefits of the current callback approach.
Since nothing else in ACS uses callbacks, all of the async methods are
essentially wrapped in synchronous calls. So nothing as it stands is
actually async.
Besides the current implementation, you need to conciser how you would
expect an implementation of the storage framework to use the callback.
The problem with callbacks is that they assume some in memory state.
This means if the process/server crashes that state is lost. Many will
say just serialize the callback to the DB, but that is very impractical.
Since ACS doesn't actually stand in the data path, an async pattern
won't really even allow it to have better performance. ACS is just
waiting for some storage operation to happen. ACS can easily spawn 1000
threads and have them all wait. If you were to get to this point, you'd
find that downstream you'll most likely have issues as you have 1000
create template operations so its killing your filer. So you will
throttle storage operations to a level that won't kill your
infrastructure and that level is no where near the scalability limits of
threads.
The callbacks pattern really complicates the code and I see no real
benefit. Instead of spending a lot of effort trying to make all of ACS
async to make it beneficial, I'd say that effort should be spent on
making ACS idempotent and crash-only. The point being, there's more
beneficial things we can do with our time.
Given that only solidfire implements the new framework (and ACS legacy
too), I would assume its a simple things for Edison to just go and
quickly change it non-async.
Darren
- [DISCUSS] Remove async APIs from storage framework Darren Shepherd
-
