You could always create a wrapper struct for the pointer and then not overload opIndex.
...alternatively, you could create a wrapper struct for the pointer, do 'alias this' on said pointer, and then overload opIndex to call static assert(0);. Like this:
struct Ptr(T) { private T *_ptr; alias _ptr this; this(T *val) { _ptr = val; } void opIndex(size_t index)() { static assert(0); } }