Xuanwo opened a new issue, #684:
URL: https://github.com/apache/opendal-reqsign/issues/684
## Problem Statement
The current Aliyun OSS implementation in `reqsign-aliyun-oss` does not
provide credential-chain parity with the most commonly used official OSS SDKs,
and the signing implementation is incomplete for modern OSS deployments.
This leads to:
- Users being unable to authenticate in common runtime environments without
custom glue code.
- Inability to adopt recommended signing versions (e.g., V4).
- Potential correctness gaps for presigned URLs when `x-oss-*` constraints
must be signed.
## Current Implementation (as of this repo state)
### Credential Chain
`services/aliyun-oss/src/provide_credential/default.rs` resolves credentials
in the following order:
1. Environment variables via `EnvCredentialProvider` (only `ALIBABA_CLOUD_*`)
2. `AssumeRoleWithOidcCredentialProvider` (via env + token file + STS)
Implemented providers:
- `EnvCredentialProvider`:
`services/aliyun-oss/src/provide_credential/env.rs`
- `AssumeRoleWithOidcCredentialProvider`:
`services/aliyun-oss/src/provide_credential/assume_role_with_oidc.rs`
- `StaticCredentialProvider`:
`services/aliyun-oss/src/provide_credential/static.rs`
Missing (commonly expected by OSS/Alibaba Cloud SDKs):
- OSS profile file (`~/.oss/credentials`) support
- Alibaba Cloud credentials/config files support (`credentials.ini`,
`~/.aliyun/config.json`)
- ECS RAM Role (metadata service) support
- `credentials_uri` support
- AssumeRole (AK-based) support (RAM role ARN + STS AssumeRole)
### Signing
`services/aliyun-oss/src/sign_request.rs` implements:
- Header signing (HMAC-SHA1, often referred to as Signature V1)
- Query signing (presigned URL) using `OSSAccessKeyId`, `Expires`,
`Signature`
Missing (commonly supported / increasingly required):
- Signature V2 (HMAC-SHA256) support
- Signature V4 support (`OSS4-HMAC-SHA256`)
Known correctness gap:
- V1 query signing path does not include canonicalized OSS headers
(`x-oss-*`) in the string-to-sign, which can be required when presigning
requests that must bind additional OSS headers.
## Goal / Success Criteria
1. Default credential chain covers the majority of official OSS SDK
credential sources without requiring custom user code.
2. Signing supports at least one modern signing version (V4), and keeps
backward-compatible versions where reasonable (V1/V2).
3. The behavior is testable with deterministic unit tests and optional
integration tests.
4. Changes remain additive and backward-compatible where possible (no
breaking API changes unless explicitly decided).
## Scope
### In Scope
- Expand env var support to include OSS SDK conventions.
- Add common credential providers and wire them into
`DefaultCredentialProvider`.
- Add Signature V4 signing support (header + presign).
- Add Signature V2 support (optional but recommended).
- Fix V1 presign header canonicalization gap (for `x-oss-*`).
- Add tests (unit + golden vectors; integration tests behind env flags).
- Align crate documentation with actual APIs and supported behaviors.
### Out of Scope (for this issue)
- Full OSS client implementation (request execution, retries, pagination,
etc.)
- Non-OSS Alibaba Cloud service signing
- Extensive refactors unrelated to Aliyun OSS credentials/signing
## Proposed Task Breakdown
### P0: High-impact credential chain parity
- [ ] Add OSS env var aliases in `EnvCredentialProvider`
- Accept `OSS_ACCESS_KEY_ID` and `OSS_ACCESS_KEY_SECRET` as aliases.
- Accept `OSS_SESSION_TOKEN` (or equivalent) as the session token alias
(confirm against official OSS SDK env set).
- Define a clear precedence order if both `ALIBABA_CLOUD_*` and `OSS_*`
are set.
- Files: `services/aliyun-oss/src/constants.rs`,
`services/aliyun-oss/src/provide_credential/env.rs`
- Tests: extend `services/aliyun-oss/src/provide_credential/env.rs` tests.
- [ ] Implement OSS profile credentials provider (`~/.oss/credentials`)
- Support `OSS_CREDENTIAL_PROFILES_FILE` override if present, else default
path.
- Support `OSS_PROFILE` (or equivalent) profile selection; default profile
when unset.
- Files: `services/aliyun-oss/src/provide_credential/profile.rs` (new),
`services/aliyun-oss/src/provide_credential/mod.rs`,
`services/aliyun-oss/src/provide_credential/default.rs`
- Tests: unit tests with in-memory file contents via `Context::file_read`.
- [ ] Implement ECS RAM Role metadata credential provider
- Fetch role name and temporary credentials from ECS metadata service.
- Cache credentials and refresh before expiration.
- Support endpoint overrides via env (aligned to Alibaba Cloud credential
chain conventions).
- Files: `services/aliyun-oss/src/provide_credential/ecs_ram_role.rs`
(new), `services/aliyun-oss/src/provide_credential/default.rs`
- Tests: mock `HttpSend` to return deterministic JSON responses.
- [ ] Implement `credentials_uri` credential provider
- Fetch credentials from `ALIBABA_CLOUD_CREDENTIALS_URI` (or equivalent).
- Cache and refresh credentials.
- Files: `services/aliyun-oss/src/provide_credential/credentials_uri.rs`
(new), `services/aliyun-oss/src/provide_credential/default.rs`
- Tests: mock `HttpSend` to return deterministic JSON responses.
- [ ] Update `DefaultCredentialProvider` chain order
- Proposed order (tentative):
1) Explicit env/static (existing)
2) OSS profile file
3) `credentials_uri`
4) ECS RAM Role metadata
5) AssumeRoleWithOIDC (existing)
6) AssumeRole (AK-based) (P1)
- Ensure each provider is individually configurable/disable-able via
builder.
- Files: `services/aliyun-oss/src/provide_credential/default.rs`
### P0: Signing correctness and modern version support
- [ ] Fix V1 presign to include canonicalized OSS headers when present
- Ensure the presign string-to-sign includes canonicalized `x-oss-*`
headers if the request contains them and presigning is requested.
- Files: `services/aliyun-oss/src/sign_request.rs`
- Tests: add deterministic unit tests that compare computed query
signature (or string-to-sign) for a request containing `x-oss-*` headers.
- [ ] Add Signature V4 support (`OSS4-HMAC-SHA256`)
- Support both header signing and presigned URL signing.
- Requires region and credential scope; define how the region is
configured (API vs env vs parse from endpoint).
- Files: `services/aliyun-oss/src/sign_request.rs` (or split into
versioned modules)
- Tests: golden vectors (prefer official examples), plus encoding edge
cases (path/query repeats).
### P1: Remaining credential sources for parity
- [ ] Add Alibaba Cloud shared credential/config file providers
- Support `credentials.ini` and `~/.aliyun/config.json` (confirm expected
schema and modes).
- Files: new providers under `services/aliyun-oss/src/provide_credential/`
- Tests: unit tests with sample files.
- [ ] Add AssumeRole (AK-based) provider
- Use base AK to call STS `AssumeRole` and cache credentials.
- Support common parameters (role session name, external ID).
- Files: new provider under `services/aliyun-oss/src/provide_credential/`
- Tests: mock STS responses and validate refresh behavior.
### P1: Signature V2 support (optional but recommended)
- [ ] Add Signature V2 support (HMAC-SHA256)
- Provide an API to select signing version (V1/V2/V4).
- Tests: golden vectors if available; otherwise cross-check against
official SDK outputs in integration tests.
### P2: Documentation and compatibility cleanup
- [ ] Align `services/aliyun-oss/README.md` and crate docs with actual APIs
- Current README references types (`Config`, `Builder`, `DefaultLoader`)
that do not exist in this crate layout.
- Update docs to reflect `RequestSigner`, `DefaultCredentialProvider`, and
the supported sources/versions.
- Files: `services/aliyun-oss/README.md`, `services/aliyun-oss/src/lib.rs`
## Validation Plan
### Unit Tests
- Provider tests:
- Env provider: alias env vars, precedence rules, token handling.
- Profile provider: profile selection, missing fields, parse errors.
- ECS metadata provider: happy path + error handling + refresh behavior.
- Credentials URI provider: happy path + error handling + refresh behavior.
### Signing Tests
- V1:
- Header signing: stable string-to-sign and Authorization formatting.
- Presign: correct query shape and signature, including `x-oss-*`
canonicalization.
- V4:
- Header signing and presign: golden vectors with fixed timestamps.
- Edge cases: percent-encoding, repeated query keys, empty query values.
### Optional Integration Tests (gated)
- If env vars such as `REQSIGN_OSS_TEST_ENDPOINT` are provided, run live
requests against a test bucket.
- Keep integration tests opt-in to avoid CI flakiness and credential leakage.
## Risks / Notes
- Introducing V4 may require new configuration (region, service scope). Keep
the default behavior backward-compatible (V1) unless users opt in.
- Metadata and URI providers involve network calls; ensure robust timeouts
and clear error messages.
- Credential caching must be concurrency-safe and must refresh before
expiration.
## References
- OSS SDK env/profile patterns (Go/Java SDK docs and implementations)
- Aliyun credential chain conventions (Alibaba Cloud credentials provider
documentation)
- OSS signing versions documentation (V1/V2/V4) and migration notes
---
**Parts of this issue were drafted with assistance from Codex (with
`gpt-5.2`) and fully reviewed and edited by me. I take full responsibility for
all changes.**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
