Theoretically exceptions are situations you expect can happen, even though the code is bug free, while assertions always indicate a bug.
I think the main implication of this is that assertions should behave differently. They should halt the program at once so the developer's attention is forced upon the bug, even though the bug may be recoverable. Furthermore, your development version of your software can run very slow, it's usually no problem. With easy way to compile out assertions, you are more inclined to use them liberally. Or to put it the other way: you are likely to implement less checking if it comes at a cost for your release build. Then there is the case where you want to leave some 'assertion checks' in release mode too, like you mention. Ideally you may want something that throws AssertError when compiled in debug mode and InvalidArgumentException in release mode. I think that whenever code is important enough to have such extensive checking, it is a good idea to do something like this.