yihua opened a new pull request, #989:
URL: https://github.com/apache/hudi-rs/pull/989

   ## Description
   
   closes #988
   
   A table open, plan and read used to build a new object store (HTTP client, 
connection pool, credential provider) at every `Storage::new` call. Now the 
table keeps the `Storage` it opened with, and derives the timeline, file system 
view, file group readers and the metadata table from it through a new 
`Storage::derive`. A derived `Storage` reuses the parent's store when the 
options are equal and the base path has the same store root: scheme and 
authority, plus the bucket or container that `object_store` takes from the 
path. It keeps its own base URL, so the object keys it reads don't change. A 
different bucket or different options still builds its own store, and nothing 
is shared across tables, because a store holds resolved credentials. The 
standalone `FileGroupReader::new_with_options` used by the bindings also builds 
one store instead of two.
   
   One case still builds a second store: the metadata table is opened without 
the data table's storage options, so it shares only when the user passed none, 
as with credentials from env vars, profiles or instance metadata. That is 
unchanged by this PR and worth its own fix.
   
   ## How are the changes test-covered
   
   - [ ] N/A
   - [x] Automated tests (unit and/or integration tests)
   - [ ] Manual tests
     - [ ] Details are described below
   
   `table_open_plan_and_read_build_one_object_store` opens a MOR table with a 
metadata table, plans, reads and computes stats, and asserts that exactly one 
store is built (it counts 9 on main). It also asserts that the metadata table 
actually opened. `new_with_options_builds_one_object_store` covers the 
standalone reader. The storage unit tests cover sharing under the same root and 
options, a new store for another bucket, scheme or option value, Azure 
containers carried in the path, and reads through a derived `Storage` at its 
own base path.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to