Re: [PR] feat: spawn-service [arrow-rs-object-store]
alamb commented on code in PR #332:
URL:
https://github.com/apache/arrow-rs-object-store/pull/332#discussion_r2045392668
##
src/client/http/connection.rs:
##
@@ -244,6 +245,67 @@ impl HttpConnector for ReqwestConnector {
}
}
+/// [`reqwest::Client`] connector that performs all I/O on the provided tokio
+/// [`Runtime`] (thread pool).
+///
+/// This adapter is most useful when you wish to segregate I/O from CPU bound
+/// work that may be happening on the [`Runtime`].
+///
+/// [`Runtime`]: tokio::runtime::Runtime
+///
+/// # Example: Spawning requests on separate runtime
+///
+/// ```
+/// # use std::sync::Arc;
+/// # use tokio::runtime::Runtime;
+/// # use object_store::azure::MicrosoftAzureBuilder;
+/// # use object_store::client::SpawnedReqwestConnector;
+/// # use object_store::ObjectStore;
+/// # fn get_io_runtime() -> Runtime {
+/// # tokio::runtime::Builder::new_current_thread().build().unwrap()
+/// # }
+/// # fn main() -> Result<(), object_store::Error> {
+/// // create a tokio runtime for I/O.
+/// let io_runtime: Runtime = get_io_runtime();
+/// // configure a store using the runtime.
+/// let handle = io_runtime.handle().clone(); // get a handle to the same
runtime
Review Comment:
I updated this example so it compiles / passes doc tests
I think it looks pretty good now:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] feat: spawn-service [arrow-rs-object-store]
alamb commented on code in PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#discussion_r2045350077 ## src/client/http/mod.rs: ## @@ -0,0 +1,41 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +//! Generic HTTP client abstraction Review Comment: Since this module is not pub its documentation is not part of the public docs. I will move the examples to the docs of `SpawnedReqwestConnector` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat: spawn-service [arrow-rs-object-store]
alamb commented on PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#issuecomment-2807251994 Note due to https://github.com/apache/arrow-rs-object-store/issues/335 the CI was not running. I plan to merge up from main shortly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat: spawn-service [arrow-rs-object-store]
ion-elgreco commented on PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#issuecomment-2801997099 @alamb I've added some docs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat: spawn-service [arrow-rs-object-store]
ion-elgreco commented on PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#issuecomment-2799938167 @alamb sure! I'll add more docs tomorrow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat: spawn-service [arrow-rs-object-store]
alamb commented on PR #332: URL: https://github.com/apache/arrow-rs-object-store/pull/332#issuecomment-2799936135 FYI @kylebarron -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] feat: spawn-service [arrow-rs-object-store]
alamb commented on code in PR #332:
URL:
https://github.com/apache/arrow-rs-object-store/pull/332#discussion_r2041114934
##
src/client/http/spawn.rs:
##
@@ -0,0 +1,162 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::client::{
+HttpError, HttpErrorKind, HttpRequest, HttpResponse, HttpResponseBody,
HttpService,
+};
+use async_trait::async_trait;
+use bytes::Bytes;
+use http::Response;
+use http_body_util::BodyExt;
+use hyper::body::{Body, Frame};
+use std::pin::Pin;
+use std::task::{Context, Poll};
+use thiserror::Error;
+use tokio::runtime::Handle;
+use tokio::task::JoinHandle;
+
+/// Spawn error
+#[derive(Debug, Error)]
+#[error("SpawnError")]
+struct SpawnError {}
+
+impl From for HttpError {
+fn from(value: SpawnError) -> Self {
+Self::new(HttpErrorKind::Interrupted, value)
+}
+}
+
+/// Wraps a provided [`HttpService`] and runs it on a separate tokio runtime
Review Comment:
It would be great to add a doc example here too (we can do this as a follow
on PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
