In reviewing the RADOS PR I ran into another question. I recently sent an email on the topic where the author wants their integration to be part of the Arrow repo (I believe this is the case for the RADOS PR). However, what about the case where the author doesn't want to be part of the Gibhub repo (so, to be clear, this email is not relevant for the RADOS PR).
Right now, in order to add a new file format to the dataset API the author has to add code to the Arrow codebase to create a new FileFormat or Fragment. Do we want to make the datasets API a "plugin" architecture to allow new formats in the future be added dynamically. Of course, now that I'm writing the email, I suppose the answer is clear. If someone cares enough about having an external extension they can always do the work to add such a plugin system. Does this sound right or is there some other reason against this or different approach we'd want to take in the future?
