flokli commented on issue #4199: URL: https://github.com/apache/arrow-rs/issues/4199#issuecomment-1991241854
> Perhaps we could make https://github.com/apache/arrow-rs/blob/master/object_store%2Fsrc%2Fparse.rs#L71 public? That'd help. We'd might still parse the URL twice, but that's less of a deal than constructing and throwing away the entire `dyn ObjectStore`. > > > But then what's the point of having the more composable cloud-specific builders in first place? > > For the use-cases where stores aren't configured by a URL?? Hmmh, I think in most applications you almost definitely want to have a combination of both. `s3://my-bucket/some-subpath` works as a pretty good identifier to specify the protocol, bucket name and subpath where something is located, so that's something people exchange in a configuration file. However, "how to get there" usually is very specific to the environment - a local developer on their local machine might have a static access key pair, or some AWS SSO config, while production workloads might use k8s and IAM roles for service accounts (env vars), or IAM roles for EC2 provided by the instance metadata server (implicit). IMHO, there's a lot of value in `object_store` having the same semantics as the "official" cloud-provider SDKs, even if the users of `object_store` use `parse_url`. Which would mean, respecting cloud-provider-specific means of configuration (env vars, ambient metadata servers) by default, and at least having the real "path" part of a URL prominently accessible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
