Note that withKeepBinary() is just a way to tell a cache not to deserialize values when doing a get or running an entry processor. The concept of binary object does not belong solely to caches - you can get an instance of IgniteBinary interface from Ignite and use binary objects in computations, for example.
For me there would be more confusion if each cache had a separate marshaller. What would then happen if you put an instance of BinaryObject to a cache with JDK marshaller? When marshaller is global, the answer is simple - BinaryObject is either available or not :)