Honestly I don't consider this a dependency problem. If you're using single components, you will have to do the wiring by yourself.
Bear in mind that this cannot be done in 'ServiceManager\Config:: configureServiceManager()' because the service manager configuration and the 'Config' service are actually not the same thing. In a full ZF2 app, the former happens to reside in the 'service_manager' key of the latter. If you look at its code, the abstract factory is just looking for a 'db' array key from different sources, 'Config' service being one of them. You could extend the factory and implement your own config loader, or just register a 'Config' service in the service manager (which you'll probably need for other components, anyway).