Gabriel39 opened a new pull request, #70493: URL: https://github.com/apache/doris/pull/70493
### Background The native MongoDB catalog sends read preference and read concern to the BE in each scan descriptor. Today those values come only from catalog properties or the connection URI, while metadata discovery uses its own primary/default policy. ### What problem does this PR solve? Issue Number: #70251 Related PR: #70327 (native reader and catalog read policies) PR Type: Feature Problem Summary: Users cannot choose MongoDB routing and visibility for an individual session or query without changing the shared catalog. Add `mongodb_read_preference` and `mongodb_read_concern`, supporting `SET` and `SET_VAR`, and pin their effective values in each scan descriptor. Both default to `inherit`, preserving existing catalog/URI behavior. Most of the diff adds tests and generated regression expectations. ### Details - **Change map:** 70 production lines changed, 281 test-source lines changed, 62 generated expectation lines, and 38 documentation lines changed. - **Scope:** Data-read policies for native MongoDB scans. Preference supports `inherit`, `primary`, `primaryPreferred`, `secondary`, `secondaryPreferred`, and `nearest`; concern supports `inherit`, `local`, `majority`, and `available`. Values are case-insensitive. Snapshot and linearizable reads remain unsupported. - **Design:** Register validated, forwarded session variables and resolve them through the existing connector session-property snapshot. Reuse the existing scan descriptor and BE collection policy application. Session overrides do not mutate catalog configuration, metadata clients, or transport pool identity. URI tags, maximum staleness, and hedge options remain effective; an incompatible primary override fails during planning without exposing the URI. - **Usage:** `SET mongodb_read_preference='secondaryPreferred'; SET mongodb_read_concern='local';` or `SELECT /*+ SET_VAR(mongodb_read_preference='primary', mongodb_read_concern='majority') */ ...`. Set either variable to `inherit` to use its catalog/URI policy again. - **Default behavior:** `inherit` independently resolves each policy from URI options first, then catalog defaults. Queries without overrides retain their existing behavior. - **Compatibility:** No connector SPI, Thrift, BE code, or storage-format changes. Metadata discovery and collection-identity verification retain their existing policies. Read concern does not establish a snapshot shared by scans. ### Testing All checks below ran locally against MongoDB 8.0.20, with the new FE and the existing ASAN native BE: - Red/green: the new scan-plan tests first failed because overrides were ignored, and the new session tests first failed because the variables were unregistered; both passed after implementation. - `run-fe-ut.sh --run` with `SessionVariableMongoReadPolicyTest`, `VariableMgrExportTest`, `MongoScanPlanTest`, and `MongoCatalogPropertiesTest`: passed. The three new session tests cover defaults, supported/unsupported values, connector snapshots, isolation, and forwarding. - `mvn -f fe/pom.xml -pl :fe-connector-mongodb -am test`: 35 connector tests passed, including URI precedence, independent policy inheritance, incompatible routing criteria, and stable transport configuration identity. - `build.sh --fe -j12`: passed. FE core and MongoDB connector Checkstyle: zero violations. - Generated `test_mongodb_read_policy.out` using the regression runner, checked its results against the synthetic fixture, then independently reran `test_mongodb_native` and `test_mongodb_read_policy`: 2 suites passed, 0 failed/fatal/skipped. Covers SET/SET_VAR, all supported modes/levels, invalid values, failed-hint restoration, URI policy inheritance, and recovery. - Additional replica-set integration: source profiler verified primary/secondary selection and the actual read concern for 8 sequential reads and 32 concurrent reads across 4 isolated sessions. Also verified URI tags with a mode override and policy restoration after a failed hint. This was an additional local acceptance check; the committed SQL suites use the existing standalone fixture. ### Release note Add `mongodb_read_preference` and `mongodb_read_concern` session variables for native MongoDB queries, with statement-level SET_VAR support. Both default to `inherit` to retain the catalog's existing read policy. ### Check List (For Author) - Behavior changed: Yes (see Details; explicit overrides now affect native MongoDB data reads). - Does this need documentation: Yes; the connector README is updated in this PR. - Pre-submission status: Ready. Local checks passed; CI is pending. ### AI Disclosure - Tool: Codex (GPT-6). - Generated: Implementation, tests, connector documentation, and this description. - Human review: Maintainer review pending; the agent completed a full diff self-review and local validation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
